Data Engineer II; AWS, Databricks, PySpark

Position: Data Engineer II (AWS, Databricks, PySpark)

Taking care of our customers, our communities and each other. That’s the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it.

Job

Category Technology Compensation  Overview

The annual base salary range provided for this position is a nationwide market range and represents a broad range of salaries for this role across the country. The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role; the skills, education, training, credentials and experience of the candidate; and other conditions of employment.

As part of our comprehensive compensation and benefits program, employees are also eligible for performance-based cash incentive awards.

Salary Range$ - $Target Openings1

What Is the Opportunity?

Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Data Engineer, you will play a key role in growing and transforming our analytics landscape. In addition to your strong analytical mind, you will bring your inquisitive attitude and ability to translate stories found in data by leveraging a variety of data programming techniques.

You will leverage your ability to design, build and deploy data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning and business intelligence/insights.

What Will You Do?

Design, develop, and maintain scalable batch and streaming data pipelines using Databricks and cloud-native services

Build and optimize ETL/ELT workflows using Python, SQL, and Spark

Work with structured and semi-structured data stored in cloud object storage (e.g., Amazon S3)

Collaborate with analytics, BI, and application teams to deliver high-quality, trusted datasets

Apply Fin Ops best practices by monitoring resource usage, optimizing job performance, and identifying cost-saving opportunities

Support AI/ML use cases by preparing and curating high-quality datasets for model training and inference

Assist with feature engineering pipelines and data validation for ML workflows

Implement data quality checks, monitoring, and performance tuning

Follow best practices for data security, governance, and access controls

Support production workloads, troubleshoot issues, and participate in operational support as needed

Contribute to documentation, standards, and reusable data engineering patterns

What Will Our Ideal Candidate Have?

Bachelor’s Degree in STEM related field or equivalent.

Eight years of related experience.

Hands-on experience with Databricks and Apache Spark

Strong experience with AWS (e.g., S3, IAM, EC2, Lambda, Glue, or related services)

Proficiency in Python and SQL

Experience working with cloud data platforms and distributed data processing systems

Solid understanding of data modeling, ETL/ELT concepts, and performance optimization

Strong communication skills and ability to work in a collaborative team environment

Highly proficient use of tools, techniques, and manipulation including Cloud platforms, programming languages, and a full understanding of modern software engineering practices.

The ability to deliver work at a steady, predictable pace to achieve commitments, deliver complete solutions but release them in small batches, and identify and negotiate important tradeoffs.

Demonstrated track record of domain expertise including understanding technical concepts necessary and industry trends, and possess in-depth knowledge of immediate systems worked on and some knowledge of adjacent systems.

Strong problem solver who ensures systems are built with longevity and creates innovate ways to resolve issues.

Strong written and verbal communication skills with the ability to work collaborate well with team members and business partners.

Ability to lead team members and help create a safe environment for others to learn and grow as engineers. and a proven track record of…