Senior Data Engineer

Role Overview :

We are seeking a Senior Data Engineer with strong developer DNA — someone who codes daily in PySpark, SQL, and Python, builds end-to-end data solutions, and leads design of robust, production-grade data platforms on Microsoft Fabric, Azure Databricks, and ADF. This is a hands-on leadership role for engineers who can think architecturally and code efficiently.

Key Responsibilities:

• Design, develop, and optimize complex data pipelines and lakehouse architectures.

• Lead development of batch and streaming ingestion frameworks using ADF, Databricks, and Event Hubs.

• Write optimized PySpark/SQL transformations for performance, scalability, and cost efficiency.

• Build reusable data frameworks and automation scripts for continuous integration and deployment.

• Establish unit testing, code review, and versioning standards across engineering teams.

• Integrate with Dev Ops CI/CD pipelines for automated releases and environment management.

• Mentor junior engineers on coding practices and architecture design.

• Collaborate with data architects on Fabric-based Lakehouse implementations.

Skills &

Qualifications:

• 6–12 years of strong hands-on experience in data engineering and development.

• Expertise in:

o PySpark, SQL, Python, and performance tuning of large-scale transformations.

o Databricks, ADF, ADLS, Fabric Data Pipelines, and Delta Lake.

• Deep understanding of data architecture, schema design, job orchestration, and data governance.
• Strong knowledge of CI/CD (Azure Dev Ops) and infrastructure-as-code for data pipelines.

• Optional:
Exposure to GCP Big Query, Dataproc, or Dataflow as secondary platforms.

Preferred

Certifications:

• Databricks Certified Data Engineer Professional
• Microsoft Certified:
Fabric Analytics Engineer Associate (DP-600)
• Microsoft Certified:
Azure Solutions Architect Expert (AZ-305)
• Optional:
Google Cloud Certified – Professional Data Engineer