Applied Scientist, AI Evaluation Platform

Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the world, changing lives for the better.

It's the diversity of our people and their thinking that inspires the innovation that runs through everything we do. When we bring everybody in, we can do the best work of our lives. Here, you'll do more than join something - you'll add something.

Our team, part of Apple Services Engineering, is looking for an Applied Scientist to lead the design and continuous development of automated benchmarking methodologies for AI-powered code assistant tools. In this role, you will investigate how coding-focused LLM agents behave, create rigorous evaluation frameworks, and establish scientific standards for assessing their quality and reliability. This role is crafted to enable the development of scalable evaluation frameworks that ensure our engineers have the right tools to create products that surprise and delight our customers.

The successful candidate will have a proactive approach with the ability to work independently and collaboratively on a wide range of projects. In this role, you will work alongside a small but impactful team, collaborating with ML and data scientists, software developers, project managers and other teams at Apple to understand requirements and translate them into scalable, reliable, and efficient evaluation frameworks.

Publications in ML evaluation or related fields.

Experience with automated testing frameworks. Experience constructing human-in-the-loop or multi-turn evaluation setups. Prior work on agentic developer tools.

Advanced degree (MS or PhD) in Computer Science, Software Engineering, or equivalent research/work experience. Strong research background in empirical evaluation, experimental design, or benchmarking. Strong proficiency in Python. Intermediate proficiency in Swift. Deep familiarity with software engineering workflows and developer tools. Experience working with or evaluating AI/ML models, preferably LLMs or program synthesis systems. Strong analytical and communication skills, including the ability to write clear reports.