Senior Associate, Azure Data Engineer

At PwC, our people in infrastructure focus on designing and implementing robust, secure IT systems that support business operations. They enable the smooth functioning of networks, servers, and data centres to optimise performance and minimise downtime. In infrastructure engineering at PwC, you will focus on designing and implementing robust and scalable technology infrastructure solutions for clients. Your work will involve network architecture, server management, and cloud computing experience.

As an Azure Data Engineer, you will play a crucial role in designing, building, and maintaining our enterprise data platform on the Microsoft Azure cloud. Your primary responsibility will be developing scalable, reliable, and efficient data pipelines to ingest, transform, and serve data to business analysts, data scientists, and operational systems. This role demands a deep understanding of modern data architecture, a passion for automation, and hands‑on expertise with the Azure data stack.

As a Senior Associate, Azure Data Engineer, you’ll work as part of a team of problem solvers, helping to solve business issues, deliver high quality client service and operational efficiency. Responsibilities include but are not limited to:

• Architect, design, and own the development of robust, end‑to‑end data pipelines for our production environment, utilizing Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
• Lead the design and implementation of advanced data models within our Azure Data Lake. Champion the Medallion Architecture (Bronze/Silver/Gold) and build complex business logic into scalable, efficient transformations using PySpark and Delta Lake.
• Develop ETL/ELT processes to ingest data from a wide variety of sources, including relational databases (SQL, Oracle), APIs, and streaming data feeds.
• Implement data models, transformations, and cleansing routines to prepare data for analytics.
• Automate data workflows and implement CI/CD practices for data pipelines using Azure Dev Ops or Git Hub Actions to ensure quality and rapid deployment.
• Monitor, troubleshoot, and optimize data solutions for performance, scalability, and cost‑effectiveness.
• Proactively identify and resolve performance bottlenecks in data pipelines. This includes hands‑on Spark job tuning, query optimization, data partitioning strategies, and implementing cost‑control measures across our Azure data services.
• Work closely with data analysts, data scientists, and business stakeholders to understand their data requirements and deliver high‑quality data solutions.
• Mentor junior data engineers through code reviews, paired programming, and technical guidance. Establish and document coding standards, best practices, and reusable frameworks to elevate the entire team’s technical capabilities.
• Partner directly with data scientists, analysts, and business leaders to translate complex business requirements into robust, scalable, and reliable technical solutions.
• Ensure data quality, integrity, and governance by implementing best practices, data validation checks, and robust documentation.

• Microsoft Azure Data Engineer Associate (DP‑203) certification is required, or the ability to obtain it within the first 2 months.
• 5-7 years of dedicated, hands‑on experience in a data engineering role, with a proven track record of delivering complex projects to production.
• Deep, hands‑on expertise and production experience with core Azure data services:
  Azure Data Factory (ADF), Azure Databricks, and Azure Data Lake Storage (ADLS Gen2).
• Advanced proficiency in Azure Data Factory, including creating dynamic, metadata‑driven pipelines, managing complex control flows and triggers, and configuring Self‑Hosted Integration Runtimes for hybrid data movement.
• Expert‑level proficiency in SQL and Python. You must have extensive, production‑level experience with PySpark for large‑scale, distributed data processing.
• Proven experience ingesting data from diverse and complex sources, including real‑time streaming data from Kafka or Azure Event Hubs, REST APIs, and files from SFTP servers.
• Proven…