Technology Lead-Java Spark

Location: Bengaluru

Key Responsibilities :
Big Data Development :
Design and implement  big data solutions  using  Spark  and  Scala . Develop and manage data pipelines and workflows within the  big data ecosystem .
Data Processing & Analysis :
Write optimized  SQL queries  for data processing and analysis. Apply data transformations and business rules using  Spark ,  Scala , and  Hadoop .
Hands-on Coding :
Develop and maintain code using  Java ,  Python , and optionally  Go Lang , ensuring the efficient processing of large datasets and integrating with other platforms and services.
Data Optimization :
Tune and optimize Spark jobs for better performance, reducing computation time and enhancing resource utilization in a big data environment.
Agile Development :
Collaborate with cross-functional teams using the  Agile methodology  to ensure timely delivery of high-quality data solutions.
Collaboration & Reporting :
Work with business analysts and other stakeholders to gather requirements, provide technical input, and ensure alignment with business needs. Regularly update stakeholders on the progress of ongoing projects.
Testing & Troubleshooting :
Conduct unit tests, troubleshoot issues, and ensure the overall quality of the data pipelines and solutions.
Requirements :
Experience : 7+ years of hands-on experience as a  Big Data Engineer , specifically with  Spark ,  Scala ,  Python , and  Hadoop .
Strong Coding Skills :
Proven expertise in coding with  Java ,  Python , and  Scala  for building scalable and efficient data processing pipelines.
SQL Expertise :
Strong knowledge of  SQL  for data querying and transformations.
Big Data Ecosystem Knowledge :
Deep understanding of the  big data ecosystem , including  Hadoop  and related tools like  Hive ,  Pig , and  HBase .
Analytical & Problem-Solving :
Strong analytical and troubleshooting skills, with the ability to find solutions to complex big data challenges.
Agile Development :
Familiarity with  Agile development methodology  and experience working in Agile teams.
Nice to Have :

Experience with  Go Lang  and familiarity with cloud-based big data platforms (AWS, GCP, Azure).
Key Technologies :
Big Data Tools :  Spark ,  Hadoop ,  Hive ,  Pig .
Languages :  Scala ,  Java ,  Python ,  SQL ,  Go Lang .
Methodology :  Agile .
Operating Systems :  Unix/Linux  for executing commands and scripts.
Work Environment :
Work Mode : [Specify WFO/WFH/Hybrid].
Location : [Specify Location].
Vendor Rate :
[Insert vendor rate in local currency].