Software Engineer , Core Data

Company Description

Etsy is the global marketplace for unique and creative goods. We build, power, and evolve the tools and technologies that connect millions of entrepreneurs with millions of buyers around the world. As an Etsy Inc. employee, whether a team member of Etsy or Depop, you will tackle unique, meaningful, and large-scale problems alongside passionate coworkers, all the while making a rewarding impact and Keeping Commerce Human.

$ - $

Etsy's Data Enablement department builds systems and infrastructure and writes data pipelines for collecting, storing, and analyzing huge sets of data in batch and streaming contexts. As a member of the Core Data team, you'll work on the pipelines that transform Etsy’s rich clickstream data sources into core datasets that are integral to thousands of downstream data processing jobs. You’ll help build out the transform layer to integrate low-latency stream processing into traditional batch jobs;

you’ll help design new methods and tooling for ensuring the validity and quality of our data; and you’ll write new robust data pipelines in Spark, Big Query SQL, and Kafka Streams.

We work in Scala, Python, Java, and SQL, and we work with technologies like Spark, Kafka, Airflow, Looker, Dataproc, and Big Query. Experience with any of these is helpful but not required; above all, we value curiosity, enthusiasm, responsibility and generosity of spirit. We're looking for a flexible engineer who cares about the data that flows through pipelines and the internal customers who will use that data.

We care more about being excited to pivot to the appropriate tool for the job than having deep expertise in single systems.

The technical staff at Etsy believes that code is craft, and that the work we do is part of a larger creative culture that includes the hundreds of thousands of inspired artists and designers who make Etsy such a unique marketplace. We believe that small, empowered, self-motivated teams can do big things. We believe in measuring everything, taking advantage of our continuous deployment system to ship code early and often, and keeping up a blameless culture based on trust and a commitment to learning.

This is a full-time position reporting to the Senior Engineering Manager, Core Data. In addition to salary, you will also be eligible for an equity package, an annual performance bonus, and our competitive benefits that support you and your family as part of your total rewards package at Etsy.

For this role, we are considering candidates based in the United States. Candidates living within commutable distance of Etsy’s Brooklyn Office Hub or in the San Francisco Bay Area may be the first to be considered. For candidates within commutable distance, Etsy requires in-office attendance once or twice per week depending on your proximity to the office. Etsy offers different work modes to meet the variety of needs and preferences of our team.

Learn more details about our work modes and workplace safety policies here.

• Our team is responsible for the daily delivery of Etsy’s core datasets, powering hundreds of business-critical datasets downstream.

• We’re constantly working to improve upon our data services and extract value from existing data sources by providing new and scalable ways to access and transform our clickstream data.

• We ensure the health and timely delivery of the ETL pipelines and data models that power our core business analytics datasets and the upstream feature data that feeds all of Etsy's machine learning systems.

• We build and maintain internal tools for monitoring and validating high-quality data.

• You will work with your team to improve our pipeline that helps us understand how buyers arrived at Etsy and their experience on the site and app.

• You will help build and maintain streaming datasets that power our search, ads, and recommendation systems.

• You will support evolving our bot filtering rules to ensure the quality of our data.

• You will partner with product engineering teams to ensure that data collected is of high quality and consistent across all of our…