Senior Machine Learning QA Engineer

Position Overview
As a Senior ML QA Engineer in the Research Enablement team, you will work side‑by‑side with researchers, ML engineers, and software engineers to define and uphold quality standards for ML systems. You are a quality‑focused engineer who is passionate about reliable, repeatable evaluation of ML models and data. Your skills span test strategy, automation, and a little MLOps, with a strong software engineering base.

You are excited to collaborate across research and product to ship ML capabilities with clear quality gates. You are comfortable working at the intersection of research and product and are competent in using Autodesk CAD software.

Reporting structure:
You will report to an Engineering Manager in Research Enablement.

Location:

Toronto, Canada (Hybrid). We are a global team, located in London, San Francisco, Toronto, and remotely. Autodesk is a hybrid‑first company, allowing workers to work remotely, in an office, or a mix of both.

Responsibilities

Define ML quality strategy and acceptance criteria across data, model, and system levels

Design and maintain model evaluation suites, metrics, and test datasets

Evaluating CAD RL model outputs for geometric validity or policy stability

Defining structured rubrics that translate qualitative findings into measurable evaluation gates

Testing ML models from product side

API testing

Automate ML QA workflows using Python and CI/CD (e.g., Git Hub Actions, Jenkins)

Create and maintain test harnesses for ML services and APIs

Mentor teams on ML QA best practices and consistent evaluation standards

Build quality gates for training and deployment pipelines (e.g., regression checks, drift detection)

Contribute to multi‑team projects and codebases, ensuring code quality and consistency

Participate in code reviews and provide constructive feedback to peers

Document and present findings and ideas across the company

Minimum Qualifications

Bachelor’s degree in Computer Science, Engineering, or equivalent experience

7+ years of professional experience in software engineering or QA for ML/AI systems

Strong programming skills in Python, with experience in test automation

Familiarity with popular CAD environments and tooling

Proficient in automation and UAT test suite/framework

Experience designing QA frameworks or platforms used by multiple teams

Excellent problem‑solving skills and attention to detail

Strong communication and collaboration skills

Understanding of software architecture and design patterns

Ability to work in an agile development environment

Preferred Qualifications

Experience with data validation tooling (e.g., Great Expectations) or labeling workflows

Familiarity with ML frameworks (e.g., PyTorch, Tensor Flow)

Experience with CI/CD tools and processes

Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)

Familiarity with MLOps practices (model monitoring, drift, deployment checks)

Experience with ML evaluation methods, metrics, and benchmarking

Passion for learning new technologies and improving existing systems

Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)

Experience testing ML services in production environments

Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)

Ideal Candidate

You demonstrate initiative to provide solutions and to learn and develop new technologies

Comfortable building QA systems from scratch and writing maintainable automation

You enjoy learning and collaborating across global locations

You are comfortable working in newly forming ambiguous areas

You are comfortable building scalable and maintainable systems that will be relied on by others

You can communicate well with others

Position Description (French)
En tant qu’ingénieur senior en assurance qualité ML au sein de l’équipe Research Enablement, vous travaillerez en étroite collaboration avec des chercheurs, des ingénieurs ML et des ingénieurs logiciels afin de définir et de maintenir les normes de qualité des systèmes ML. Vous êtes un ingénieur axé sur la qualité, passionné par l’évaluation fiable et reproductible des modèles et des données ML.

Vos compétences couvrent la stratégie de test, l’automatisation et un…