Senior Data Engineer

Overview

Senior Data Engineer (active SC clearance required) —

Location:

UK (Remote, occasional London travel) | Permanent: £60-85K

Hands-on Data Engineer working with public sector clients to design, build, and operate production data platforms. Lead technical architecture across cloud environments, mentor delivery teams, and embed analytics/AI capabilities in agile, client-facing projects.

• Translate complex business challenges into scalable data pipelines, models, and analytics solutions
• Build/operate production-grade pipelines using SQL/Python/Spark across lake houses and event-driven systems
• Architect semantic models and data visualisations for business intelligence platforms
• Deploy cloud data/AI solutions (AWS/Azure) with robust CI/CD, testing, and security practices
• Integrate ML models, experimentation frameworks, and governance workflows
• Deliver client-ready dashboards, insight narratives, and consulting materials
• Collaborate across multidisciplinary teams in agile, iterative delivery cycles

• Core
  :
  Advanced SQL/Python, PySpark/Pandas, Spark orchestration tools
• Cloud
  : AWS (Databricks/Redshift equivalents), Azure analytics platforms, Git/CI-CD
• Architecture
  :
  Data lake houses, event streaming, semantic modelling for BI tools
• AI/ML
  :
  Model APIs, prompt engineering, experimentation frameworks
• Governance
  :
  Data catalogues, lineage tracking, security/access controls, GDPR

• Expert SQL/Python for data modelling, transformation, and automation
• Data architecture experience: lake houses, event-driven systems, BI semantic models
• Software engineering:
  Git workflows, CI/CD pipelines, testing frameworks
• Analytics: experimentation design, statistical analysis, insight storytelling
• ACTIVE SC clearance required
• Advanced BI semantic modelling (Power BI/Tableau equivalents)
• Operational intelligence platforms or data mesh implementations
• Data governance frameworks (lineage/catalogues/access management)