Azure Databricks Architect

Azure Databricks Architect

6 months

Remote with 2 days travel to London per month

Active SC

Inside IR35

We require an Azure Databricks Architect to provide architectural leadership and assurance for the design, implementation, and ongoing evolution of an Azure-based data and analytics platform within a public sector environment. The role will focus on platform architecture, data patterns, governance, and non‑functional requirements, working alongside delivery teams and stakeholders to ensure solutions are secure, scalable, and compliant with organisational and government standards.

• Define and maintain the end‑to‑end architecture for Azure Databricks‑based data platforms. Design Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), and associated Azure services. Produce and maintain High‑Level Designs (HLDs) and Low‑Level Designs (LLDs). Establish and govern data ingestion, transformation, and consumption patterns.
• Design and oversee data security and access models, including role‑based access control and data segregation. Ensure alignment with public sector governance, security, and compliance requirements. Address non‑functional requirements including performance, resilience, scalability, and cost management. Support architectural assurance activities and formal design review processes.
• Define integration patterns with Azure services, including Azure Data Factory, event‑based and batch ingestion services, and downstream analytics and reporting platforms. Provide architectural guidance on CI/CD and Infrastructure‑as‑Code approaches. Support platform optimisation and sustainable operating models.
• Act as a design authority for Azure Databricks architectural decisions. Provide guidance and oversight to delivery teams, including data engineers. Support early‑phase discovery and design for new data and analytics use cases. Identify architectural risks, constraints, and dependencies and support mitigation planning.

Demonstrable experience designing Azure Databricks architectures in enterprise environments. Strong understanding of Databricks Lakehouse architecture, Delta Lake, and governance concepts. Architectural‑level knowledge of Spark‑based platforms. Experience designing secure, governed data platforms in regulated or public sector environments. Proven ability to produce HLDs, LLDs, and architectural decision documentation. Strong knowledge of Azure data and integration services.

Experience delivering data platforms within UK public sector or government organisations. Familiarity with government cloud and security standards. Experience supporting architectural assurance or design review boards. Knowledge of enterprise analytics and reporting platforms. Relevant Azure or Databricks certifications.

The role will operate in an architectural and advisory capacity with limited hands‑on delivery. The architect will work across multiple teams and work streams rather than being embedded in a single delivery team. The role will support both delivery and assurance activities throughout the lifecycle of the platform.