Senior Data Engineer

Key Responsibilities:

• Design, develop, and maintain scalable, real-time and batch data pipelines using AWS Glue, Lambda, Apache Spark, and Kafka.
• Implement data lake architecture to ensure efficient data storage, processing, and retrieval.
• Collaborate with cross-functional teams to understand data requirements and ensure the data infrastructure supports their needs.
• Use Terraform to manage and provision infrastructure in a reproducible and scalable manner.
• Optimize and troubleshoot complex data pipelines, ensuring high availability and performance.
• Explore and integrate the latest technologies to enhance our data processing capabilities.
• Work in a fast-paced, startup-like environment where you will take ownership of key projects and contribute to our overall data strategy.

Technical Requirements:

• 3 - 5 years of experience as an intermediate engineer with proficiency in AWS services.
• Strong programming skills in python and SQL.
• Strong experience with Apache Spark and Delta Lake for big data processing.
• Expertise in using Terraform for Infrastructure as Code (IAAC).
• Proficiency using standard Dev Ops tool such as Github, Azure Dev Ops, etc.
• Experience with Kafka and real-time data streaming pipelines along with geospatial data processing and analysis will be good to have.

Why Join Us?

• Work with complex and large datasets that will challenge and expand your skill set.
• Be part of a startup-like environment where your ideas and contributions directly impact the company's success.
• Take ownership of projects and have the autonomy to drive them from inception to completion.
• Grow your career in a fast-paced environment with plenty of opportunities to learn and work with the latest technologies.
• Collaborate with a passionate and innovative team that values your input and encourages professional growth.