Lead Data Engineer

My client are seeking an experienced Lead Data Engineer to lead the migration of customer data from a third‑party platform onto the new Databricks environment. This is a hands‑on leadership role requiring deep Databricks expertise combined with architectural ownership and delivery accountability.

• Lead the migration of customer data from an external third‑party provider to the new Databricks platform.
• Define and implement the end‑to‑end data architecture strategy within Databricks.
• Design scalable, secure, and high‑performance data pipelines.
• Establish governance, data modelling standards, and best practice frameworks.
• Act as technical authority and SME for Databricks across the Programme.
• Provide architectural oversight while remaining hands‑on with engineering delivery.

• Deep hands‑on expertise in Databricks (architecture, optimisation, deployment).
• Deep expertise in Azure data services (Databricks, Data Factory, ADLS) and strong understanding of distributed data processing.
• Hands‑on experience with CI/CD tooling (Azure Dev Ops/Git Hub) and modern Dev Ops practices for data engineering.
• Strong grasp of data architecture, ETL/ELT design, and performance optimisation.
• Proven experience leading enterprise data platform implementations.
• Experience operating in senior/lead/senior/lead/principle‑level engineering roles.
• Strong understanding of data governance, security, and compliance frameworks.
• Comfortable leading small, high‑performing teams.
• Experience working within retail, eCommerce, or customer‑focused environments (preferred).