Data Scientist

About Us:

At Sauce Labs, we empower the world's top enterprises - like Walmart, Bank of America, and Indeed - to deliver quality web and mobile applications  industry-leading platform ensures continuous quality across the SDLC, using AI-powered analytics to identify key quality signals from development through production. With our unified solution, teams can release and innovate with confidence, knowing their apps will always look, function, and perform exactly as they should.

Backed by TPG and Riverwood Capital, we are shaping the future of digital confidence - join us!

The Role:

At Sauce Labs, we’re looking for a  Data Scientist / GenAI Engineer  to join our team and work directly with our engineering crew on the next generation of AI-powered products. You’ll be right in the mix of building, evaluating, and refining our new AI Assistant, helping our customers unlock deeper, smarter insights from their testing data. If you love collaborating across teams to turn complex data into helpful AI features, we’d love to meet you!

Responsibilities:
Collaborate with the engineering team to execute experiments and provide insights
Prompt engineering and optimization for accuracy, relevance, and hallucination reduction
Research new use cases for AI-powered features
Monitor the accuracy of AI solutions over time
Collect and analyze data across Sauce Labs
Manage the data directory across Sauce Labs - work with the data engineering team
Analyze time-series testing datasets to identify patterns and insights
Analyze telemetry data for performance and usage patterns
Analyze logs and traces for root cause analysis
Discover actionable insights from the data
Evaluate model performance using GenAI evaluation frameworks
Design and maintain golden datasets for GenAI evaluation
Build evaluation pipelines using MLflow and LLM-as-judge frameworks
Develop deterministic and LLM-based scoring rubrics for answer validation

Required Skills:

Strong Python skills (Pandas, data manipulation, LLM frameworks)

Experience with GenAI evaluation metrics (recall@k, MRR, faithfulness, F1)
Proficiency in prompt engineering (few-shot, grounding, structured outputs)
Familiarity with RAG techniques (hybrid retrieval, re-ranking, chunking strategies)
SQL proficiency (Snowflake or Postgre

SQL)
Understanding of LLM-as-judge evaluation and scoring rubrics
Knowledge of data governance (bronze/silver/gold data tiers)

Experience with experiment tracking tools (MLflow, Weights & Biases, Lang Smith)

Experience with agentic frameworks (MCP, tool calling, ReAct patterns)
Nice to Have:
Knowledge of fine-tuning techniques (SFT, LoRA, DPO)
Familiarity with vector databases (Pinecone, Weaviate, Chroma)
Understanding of LLM security (prompt injection defense, tool safety)

Experience with advanced RAG (Graph-RAG, Self-RAG, Corrective RAG)
Knowledge of Snowflake Cortex AI features
Please note our   privacy terms   when applying for a job at Sauce Labs.
Sauce Labs is proud to be an Equal Opportunity employee and values diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender identity/expression/status, sexual orientation, age, marital status, veteran status or disability status.
Security responsibilities at Sauce
At Sauce, we will commit to supporting the health and safety of employees and properties, partnering with internal stakeholders to learn and act on ever-evolving security protocols and procedures. You’ll be expected to fully comply with all policies and procedures related to security at the department and org wide level and exercise a ‘security first’ approach to how we design, build & run our products and services.