Lead Data Engineer

Are you a skilled Data Engineer looking to step into the world of architecture?

An exciting opportunity has opened for an experienced Data Engineer to join our national Data & Analytics function at a time of significant technical modernisation. The team is about to embark on a greenfield project to build a futureproof data warehousing platform using Azure Data Factory and Databricks, with a clear focus on scalability, quality, and best practice.

Working within a specialist Data Platform & Engineering team, you’ll join as the expert within Azure Databricks and PySpark. You'll help to upskill our teams knowledge of Databricks, get involved in data modelling and ETL pipeline development, integrations, and performance tuning. This is an ever evolving project as the business becomes more and more data driven and your knowledge of Databricks will be pivitol in shaping the architecture.

This is an ideal role for a Lead Data Engineer ready to step into an architectural role, or an established architect keen to take ownership of a highly visible and strategic data platform project.

• Leading the design and implementation of a new Databricks-based data warehousing solution
• Designing and developing data models, ETL pipelines, and data integration processes
• Large scale data processing using Py Spark
• Monitoring, tuning, and optimising data platforms for reliability and performance
• Up...