QA Engineer, Data Science

Design and perform end-to-end validation of machine learning models and Agentic GenAI solutions, independently assessing methodology, data usage, modeling techniques, assumptions, and outcomes to ensure adherence to data science best practices, governance standards, and intended business objectives across diverse datasets and scenarios.

Validate Agentic GenAI solutions, including:

Testing agent workflows, tool usage, orchestration logic, and decision paths.

Evaluating LLM outputs for accuracy, consistency, bias, hallucination risk, and alignment

with business objectives.

Assessing guardrails, prompt strategies, fallback mechanisms, and failure handling.

Create statistical robustness and performance tests for trained models and GenAI systems, including simulated datasets that reflect real-world and edge-case scenarios.

Develop automated evaluation frameworks to measure model and agent quality, including precision, recall, stability, drift, response quality, and business KPIs.

Define and track data quality and model quality metrics, including input data validation, feature integrity, and output reliability.

Test and validate machine learning models and Agentic GenAI solutions using Python and

related tools, analyze outputs and behaviors across scenarios, produce detailed validation reports, debug issues, and propose corrective actions to address methodological, data, or implementation gaps.

Collaborate with Data Scientists to deeply understand data pipelines, feature engineering, models, prompts, and agent architectures.

Work with Business SMEs to ensure models and GenAI agents meet stated objectives and risk tolerances.

Continuously seek better ways to test, validate, and monitor solutions through automation, new tools, and emerging technologies.

Bachelor’s degree in computer science, Computational Science, Engineering, or a related field.

Strong Quality Assurance experience, preferably in data, analytics, or software-intensive environments.

Solid foundation in statistics, statistical testing, and experimental design (e.g., controlled

trials, A/B testing).

Experience validating machine learning models and working with complex datasets.

Experience validating GenAI / LLM-based systems, including prompt testing, output evaluation, and agent workflows.

Working knowledge of big data environments and data pipelines.

Strong programming skills (e.g., Python) for test development, automation, and debugging.

Demonstrated passion for simplifying and automating work, continuous learning, solving open-ended problems, and improving efficiency.

Excellent communication and organizational skills, with the ability to collaborate across technical and business teams and manage multiple priorities.

Experience with distributed analytics platforms (e.g., Hadoop, Spark).

Familiarity with model risk management, fairness, explainability, and governance concepts.

Experience with Information Security or secure-by-design principles.

Experience working in an Agile environment.

Exposure to monitoring model or GenAI performance in production (drift, degradation, feedback loops).