Senior Data Engineer

Senior Data Engineer

Department: Technology:
Data Technology

Employment Type: Permanent - Full Time

Location: London or Manchester UK

About us

We're Transform. Transform is a fresh alternative to Big Consulting. As an end‑to‑end transformation partner, we define, design, build and operate systems that solve our clients’ toughest challenges and seize their biggest opportunities. That often means combining strategy, data, AI, technology, and change management. Sometimes we manage and operate the systems we create; other times, we hand it over for the client to run – standing by to make improvements when things change.

It all comes down to making an impact: improving lives, supporting citizens, streamlining workflows, and making the world a better place. We specialise in public sector and public‑facing organisations—from government agencies and local authorities to energy, utilities, transportation and healthcare. We also work for some of the world’s most dynamic brands across automotive, retail, financial services and consumer goods.

In 2024, Transform acquired Cadence Innova, the UK’s leading public sector change consultancy. The result is a fresh kind of partner: together, we’re able to deliver change from the very first ideas through to deployment—building on our deep relationships at every level of government.

We're part of the Next 15 group, a multi‑national organisation spanning over 25 brands. We combine the financial robustness and breadth of services of a large organisation with the nimbleness and innovation of a smaller brand.

If you are keen to help us build the workforce of the future, we should talk. We’re all in! That's what gets us out of bed every day. We hope you feel the same.

The Senior Data Engineer will be responsible for working as part of an Agile delivery squad with other data engineers and senior data engineers, delivering data engineering on a range of data products for use in insight, reporting and analytics including AI. You will work closely with product owners, business analysts, data architects, data analysts, delivery project managers and data scientists.

• Managing quality of own and teams’ outputs
• Delivering data engineering on a range of complex marketing and research processes
• Working with clients to understand needs, and best implement that with our available technology solutions
• Working with a range of data sources such as CRM, sales, research and HR data
• Building Single Customer Views, CDPs or Data Warehouses/Data Lakes to merge disparate data sets for marketing or sales activation
• Building and configuring data ingestion methods from APIs, flat files, streaming, public data sources and survey data

• Must have experience of using Azure Databricks and Medallion architecture
• Expert experience with Azure Data Factory or Python/Airflow and a range of associated libraries for data processing and manipulation
• Experience with a range of database technologies including data lakes such as Azure SQL, SQL Server, Azure Data Lake, Postgre
  
  SQL, Snowflake, MySQL, Cosmos DB or Mongo DB
• Understanding of Azure Dev Ops and CI/CD processes
• Knowledge of data warehouse or data lakes with SCVs or in conjunction with CDP platforms
• Knowledge of data ingestion of streaming data using Databricks and Pyspark
• Understanding and application of statistical methods
• Ability to lead, and to work well as part of, a team using the agile methodology

• Palantir Foundry (Pipeline Builder) experience
• Knowledge of ELT/ETL tools such as Informatica DEI/BDM and Talend would be beneficial
• Experience of mentoring junior to mid‑level Data Engineers would be beneficial

• Being one team; looking out for each other as a team
• The world never stops changing and we’ll never stop learning
• Infinity & Beyond; we’re crazy ambitious, not afraid to fail along the way (it’s just falling with style)
• Integrity is everything; we’re honest with each other and with our clients
• Diversity is our superpower; everyone’s story is different, and we celebrate differences
• We’re all in; we take on tough, complex challenges with lots of moving parts and we’re committed…