Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)

<div><span data-contrast="auto">Shield AI is a venture-backed defense-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include Hivemind autonomy software, V-BAT and X-BAT&nbsp;aircraft, and Aechelon simulation and synthetic reality technologies. With offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, Shield AI’s technology actively supports operations worldwide. For more information, visit&nbsp;</span><a rel="noreferrer noopener" href="https://shield.ai/"><span data-ccp-charstyle="Hyperlink">www.shield.ai</span></a><span data-contrast="auto">. Follow Shield AI on&nbsp;</span><a rel="noreferrer noopener" href="https://www.linkedin.com/company/shield-ai"><span data-ccp-charstyle="Hyperlink">LinkedIn</span></a><span data-contrast="auto">,&nbsp;</span><a rel="noreferrer noopener" href="https://x.com/shieldaitech"><span data-ccp-charstyle="Hyperlink">X</span></a><span data-contrast="auto">,&nbsp;</span><a rel="noreferrer noopener" href="https://www.instagram.com/shield_ai"><span data-ccp-charstyle="Hyperlink">Instagram</span></a><span data-contrast="auto">, and&nbsp;</span><a rel="noreferrer noopener" href="https://www.youtube.com/@Shield_AI"><span data-ccp-charstyle="Hyperlink">YouTube</span></a><span data-contrast="auto">.</span><span data-ccp-props="{&quot;335559685&quot;:720}">&nbsp;</span></div><div><br></div><h3>Job Description:</h3> <div> <p data-renderer-start-pos="16">The Sr. Staff Platform / Data Reliability Engineer is responsible for making Databricks a reliable, secure, scalable, and operationally mature platform for enterprise use. This role focuses on the platform itself rather than domain-specific business modeling, ensuring the lakehouse can support multiple domains, regulated data handling, and a growing set of production workloads without becoming fragile, expensive, or hard to govern.&nbsp;</p> <p data-renderer-start-pos="454">This role partners closely with the Senior Staff Data Engineer, who focuses more heavily on ingestion and medallion implementation patterns, and with the existing Cloud &amp; Infrastructure team, which owns cloud accounts, networking, and foundational infrastructure. The Platform / Data Reliability Engineer owns the Databricks operational layer that sits above that foundation: reliability, observability, deployment standards, compute and job policies, and platform enablement patterns for internal users.&nbsp;&nbsp;</p> </div><h3>What you'll do:</h3><div> <li>Own operational excellence for the Databricks platform, including monitoring, alerting, observability, incident response support, and production runbook patterns for data jobs and platform services.&nbsp;</li> <li>Define and maintain CI/CD and promotion standards for Databricks assets, including workflows, jobs, notebooks, code packages, infrastructure configuration, and environment promotion from dev to prod.&nbsp;</li> <li>Design and maintain platform standards for job orchestration, cluster and compute policies, service principal usage, environment isolation, and production execution reliability. &nbsp;</li> <li>Establish reusable operational templates and enablement patterns for new domains onboarding to Databricks, including logging conventions, job tagging, metadata capture, and support handoff expectations. &nbsp;</li> <li>Partner with the Senior Data Engineer to ensure ingestion and medallion patterns are implemented in a way that is observable, recoverable, cost-aware, and secure in production.&nbsp;</li> <li>Work with the cloud/infrastructure team to align Databricks configuration and usage patterns with broader enterprise cloud standards, especially where commercial and future government-hosted environments are involved. &nbsp;</li> <li>Help enforce technical controls for data segregation, access boundaries, and operational compliance in a highly regulated environment. &nbsp;</li> <li>Track and improve platform health metrics such as job success rates, incident trends, data pipeline reliability, cost efficiency, and environment drift.&nbsp;</li> <li>Document platform standards, operational expectations, and support models so the Databricks platform can scale beyond a small founding team. &nbsp;</li> <li>Mentor internal engineers who are growing into platform responsibilities, helping expand Databricks operational knowledge within the team. &nbsp;</li> </div><h3>Required qualifications:</h3><div> <li>12+ years of relevant experience in data platform engineering, platform operations, site reliability engineering, or modern cloud data infrastructure. &nbsp;</li> <li>Hands-on experience with Databricks or a closely related cloud data platform in production environments.&nbsp;</li> <li>Experience designing or operating CI/CD, environment promotion, version control, and deployment automation for data platforms and pipelines. &nbsp;</li> <li>Strong understanding of platform operations concepts such as observability, monitoring, alerting, incident management, and reliability engineering. &nbsp;</li> <li>Experience with compute policy design, workload isolation, service principals, and secure production execution patterns on cloud data platforms. &nbsp;</li> <li>Ability to work effectively in a regulated or security-sensitive environment with strong expectations around access control, auditability, and operational discipline. &nbsp;</li> <li>Strong collaboration skills and comfort partnering with cloud/infrastructure, security, data engineering, and analytics stakeholders. &nbsp;</li> </div><h3>Preferred qualifications:</h3><div> <li>Databricks certification and/or strong demonstrated expertise with Delta Lake, Unity Catalog, Workflows, and Databricks Asset Bundles. &nbsp;</li> <li>Experience with infrastructure-as-code and platform automation in enterprise environments. &nbsp;</li> <li>Experience supporting commercial and government or otherwise segregated environments with different compliance and access requirements. &nbsp;</li> <li>Experience in defense, aerospace, federal, or another regulated industry. &nbsp;</li> </div>