Data Engineer

Location: Bengaluru

We are seeking a  Data Engineer  to design and implement scalable data pipelines for feature extraction, transformation, and loading (ETL). You will be responsible for governing and optimizing these pipelines to ensure high reliability and data quality across both on-premise and cloud environments. This role requires strong proficiency in  Pyspark and Scala , experience with big data technologies, and proven leadership skills to guide a team of data engineers.

Roles & Responsibilities :
Data Pipeline Design & Implementation :
Design and implement scalable data pipelines for feature extraction, transformation, and loading ( ETL ) using technologies such as  Pyspark, Scala , and relevant big data frameworks.
Data Governance & Optimization :
Govern and optimize data pipelines to ensure high reliability, efficiency, and data quality across on-premise and cloud environments.
Cross-functional Collaboration :
Collaborate closely with data scientists, ML engineers, and business stakeholders to understand data requirements and translate them into robust technical solutions.
Best Practices & Compliance :
Implement best practices for data governance, metadata management, and compliance with regulatory requirements.
Team Leadership :
Lead a team of data engineers, providing technical guidance, mentorship, and fostering a culture of innovation and collaboration.
Continuous Improvement :
Stay updated with industry trends and advancements in big data technologies, contributing to the continuous improvement of data engineering practices.

Skills Required:

Strong experience in data engineering with hands-on experience in designing and implementing data pipelines.
Strong proficiency in programming languages such as  Pyspark and Scala , with experience in big data technologies ( Cloudera, Hadoop ecosystem ).
Proven leadership experience in managing and mentoring a team of data engineers.
Excellent communication skills with the ability to collaborate effectively across teams and stakeholders.
Experience working in a banking or financial services environment is a plus.

QUALIFICATION:

Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field, or equivalent practical experience.