Azure Databricks Lead

Description - External

We are Cognizant AIA - Artificial Intelligence and Analytics !!

Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all intelligent, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to design the future—a future in which trial-and-error business decisions have been replaced by informed choices and data-supported strategies!

By applying AI and data science, we help leading companies to prototype, refine, validate, and scale their AI and analytics products and delivery models! Cognizant’s AIA practice takes insights that are buried in data and provide businesses with a clear way to transform how they source, interpret, and consume their information. Our clients need flexible data structures and a streamlined data architecture that quickly turns data resources into informative, meaningful intelligence.

The Azure Databricks Lead is responsible for architecting, designing, and leading the development of scalable data engineering solutions on Azure using Databricks, Spark, and the Lakehouse ecosystem. This role provides technical leadership, oversees delivery, and ensures high‑quality, reliable data pipelines that support analytics, AI, and enterprise data modernization initiatives.

• Lead architecture, design, and development of data pipelines using Azure Databricks
  , Py Spark ,
  Spark SQL
  , and Delta Lake
  .
• Define and enforce best practices for coding, performance optimization, data quality, and CI/CD.
• Guide teams in implementing batch and streaming pipelines using Databricks and Azure services.
• Integrate Databricks with ADLS
  , Azure Data Factory
  , Synapse
  , Key Vault
  , Event Hub/Kafka
  , and other cloud components.
• Review solution designs, notebooks, jobs, and deployment pipelines; ensure adherence to standards.
• Collaborate with architects, product owners, and business stakeholders to translate requirements into scalable solutions.
• Troubleshoot complex issues, optimize compute usage, and ensure platform reliability and cost efficiency.
• Mentor developers, conduct code reviews, and support Agile delivery processes.

• Strong hands‑on expertise with Azure Databricks
  , Py Spark ,
  Spark SQL
  , and Delta Lake
  .
• Deep understanding of distributed computing, data partitioning, and performance tuning.
• Experience with Azure Data Factory
  , ADLS
  , Azure Dev Ops
  , and CI/CD for data engineering.
• Strong proficiency in Python
  , SQL
  , and data transformation frameworks.
• Experience designing Lakehouse architectures and medallion‑layered data models.
• Excellent communication, leadership, and problem‑solving skills.

Applications will be accepted until February 27th 2026. The annual salary for this position is between $120- $130K depending on experience and other qualifications of the successful candidate.

Please note, this role is not able to offer visa transfer or sponsorship now or in the future.

This position is also eligible for Cognizant’s discretionary annual incentive program, based on performance and subject to the terms of Cognizant’s applicable plans.

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability Paid Parental Leave
• Employee Stock Purchase Plan

The salary, other compensation, and benefits information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.