We are looking for a highly skilled Azure Data Engineer for an engagement to build scalable, production-ready data solutions.
The ideal candidate will have strong hands‑on experience with Python, PySpark, Azure Databricks, and Azure Data Factory, along with a solid understanding of modern data engineering best practices.
Key ResponsibilitiesDesign, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, and Spark SQL
Build and optimize workloads on Azure Databricks (notebooks, job clusters, workflows)
Implement Delta Lake best practices (schema evolution, partitioning, performance tuning, merge strategies)
- Develop robust ingestion frameworks integrating with Azure Data Lake Storage Gen2 and Azure SQL
- Implement data validation, transformation logic, and quality control frameworks
- Ensure performance optimization and cost‑efficient execution
- Support production deployments, monitor pipelines, and resolve incidents
- Maintain version control using Git and follow CI/CD best practices
- Collaborate with architects, analysts, and business stakeholders
Strong hands‑on experience in PySpark and Spark SQL
Proven experience in Spark performance tuning & optimization
Strong knowledge of Delta Lake architecture
- Experience working with Azure Data Lake Storage Gen2
- Git‑based version control experience
- Production support and troubleshooting experience
Exposure to data governance, data security, and compliance frameworks
- Experience integrating data from SAP ERP systems
- Knowledge of Azure Data Factory (ADF)
- Understanding of Dev Ops and CI/CD pipelines
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).