Java +AWS Data Engineer+Spark

TCS is Hiring Java +AWS Data Engineer+Spark for Bangalore and Hyderabad location

Experience : 8 - 12 years

Mode of interview:
Virtual

Location :
Bangalore, Hyderabad

JOB DESCRIPTION:

Must have:

- Strong proficiency in Apache Spark for big data processing, including Spark SQL, Data Frames, and RDDs.
- Solid experience in Java programming for developing data-intensive applications and Spark Jobs.
- Extensive experience with AWS cloud services for data engineering (e.g S3, EMR, Glue, Redshift, Lambda, Kinesis)
- Proficiency in SQL for data manipulation and querying

- Experience with data warehousing concepts and technologies
- Experience/Knowledge in Apache Kafka for real-time data streaming
- Nice to have: knowledge in Scala, Terraform, Python programming skill, Databricks

Responsibilities:

- Design, develop and maintain scalable and robust data pipelines using Apache Spark and Java on AWS
- Implement ETL/ELT processes to ingest, transform, and load large datasets from various sources into data warehouse on AWS(e.g S3, Redshift)
- Develop and Optimize Spark applications using Java to process and analyze large Java to process and analyze large volume of data
- Design and implement data models for efficient storage and retrieval in AWS services like Amazon Redshift, Dynamo DB, or Aurora
- Utilize various AWS services (e.g, EMR, Glue, Lambda, S3, Redshift, Kinesis) to build and manage data solutions.
- Ensure data security, compliance and cost effectiveness of data solutions within the AWS ecosystem.
- Monitor and troubleshoot data pipelines and AWS resources to ensure optimal performance and reliability.