Data Engineer

Our Company

Creospan is a subsidiary of Creospan Inc., our parent company, headquartered in Chicago, IL. From our humble beginnings in 1999 – with just a handful of employees and a mission to help our clients leverage emerging web technologies to build next-generation products – technology has changed dramatically, yet our curiosity has remained constant. Our expertise spans across Telecom, Technology, Manufacturing, Ecommerce, Insurance, Banking, Transportation, and Healthcare domains.

To find out more about us, visit our website:(Use the "Apply for this Job" box below)..com

Job Title:

Azure Databricks Engineer

Experience:

6–8 Years

Location:

Pune, India (Hybrid)

Employment Type:

Full Time

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks

• Build and optimize batch and streaming ETL/ELT workflows using Apache Spark

• Develop data processing solutions using PySpark, Spark SQL, and Delta Lake

• Integrate data from multiple Azure sources such as Azure Data Lake Storage (ADLS Gen2), Azure

SQL, and APIs

• Optimize Spark jobs for performance, cost efficiency, and reliability on Azure

• Implement data quality, validation, and reconciliation frameworks

• Collaborate with data architects, analysts, and business stakeholders

• Ensure adherence to data security, governance, and compliance standards

• Support production deployments and troubleshoot pipeline failures

Required

Skills & Qualifications

• 6–8 years of experience in Data Engineering

• Strong hands-on experience with Azure Databricks (mandatory)

• Deep expertise in Apache Spark, PySpark, and Spark SQL

• Experience working with Azure Data Lake (ADLS Gen2)

• Hands-on experience with Azure Data Factory (ADF) for orchestration

• Strong SQL skills and experience with relational databases

• Experience with CI/CD pipelines (Azure Dev Ops preferred)

• Working knowledge of cloud security and access controls on Azure