QA Engineer, Data Science

Job Description
What is the opportunity?
As the Senior/Lead QA Engineer, Data Science, you will apply quality assurance expertise, statistics, and machine learning knowledge to validate the robustness, reliability, and business effectiveness of advanced analytics, machine learning, and Agentic GenAI solutions developed by the team.

You will partner closely with Data Scientists, Engineers, and Business SMEs to understand business objectives and translate them into comprehensive validation strategies, test frameworks, and quality metrics. Your work will ensure that ML models and GenAI agents behave as expected across diverse real‑world scenarios, meet stakeholder and governance expectations, and deliver measurable business value.

In addition to testing and validation, you will play a key role in improving model quality, strengthening QA processes, and introducing automation and best practices across the Data Science and GenAI lifecycle.

What will you do?

Design and perform end‑to‑end validation of machine learning models and Agentic GenAI solutions, independently assessing methodology, data usage, modeling techniques, assumptions, and outcomes to ensure adherence to data science best practices, governance standards, and intended business objectives across diverse datasets and scenarios.

Validate Agentic GenAI solutions, including:

Testing agent workflows, tool usage, orchestration logic, and decision paths.

Evaluating LLM outputs for accuracy, consistency, bias, hallucination risk, and alignment with business objectives.

Assessing guardrails, prompt strategies, fallback mechanisms, and failure handling.

Create statistical robustness and performance tests for trained models and GenAI systems, including simulated datasets that reflect real‑world and edge‑case scenarios.

Develop automated evaluation frameworks to measure model and agent quality, including precision, recall, stability, drift, response quality, and business KPIs.

Define and track data quality and model quality metrics, including input data validation, feature integrity, and output reliability.

Test and validate machine learning models and Agentic GenAI solutions using Python and related tools, analyze outputs and behaviors across scenarios, produce detailed validation reports, debug issues, and propose corrective actions to address methodological, data, or implementation gaps.

Collaborate with Data Scientists to deeply understand data pipelines, feature engineering, models, prompts, and agent architectures.

Work with Business SMEs to ensure models and GenAI agents meet stated objectives and risk tolerances.

Continuously seek better ways to test, validate, and monitor solutions through automation, new tools, and emerging technologies.

What do you need to succeed?
Must-have

Bachelor’s degree in computer science, Computational Science, Engineering, or a related field.

Strong Quality Assurance experience, preferably in data, analytics, or software‑intensive environments.

Solid foundation in statistics, statistical testing, and experimental design (e.g., controlled trials, A/B testing).

Experience validating machine learning models and working with complex datasets.

Experience validating GenAI / LLM‑based systems, including prompt testing, output evaluation, and agent workflows.

Working knowledge of big data environments and data pipelines.

Strong programming skills (e.g., Python) for test development, automation, and debugging.

Demonstrated passion for simplifying and automating work, continuous learning, solving open‑ended problems, and improving efficiency.

Excellent communication and organizational skills, with the ability to collaborate across technical and business teams and manage multiple priorities.

Nice-to-have

Experience with distributed analytics platforms (e.g., Hadoop, Spark).

Familiarity with model risk management, fairness, explainability, and governance concepts.

Experience with Information Security or secure‑by‑design principles.

Experience working in an Agile environment.

Exposure to monitoring model or GenAI performance in production (drift, degradation, feedback loops).

What's in it for you?
We thrive on the challenge to be our best, progressive…