Member of Technical Staff - Copilot AI Evaluation Engineering Manager

Overview

As Microsoft continues to push the boundaries of AI, we are on the lookout for passionate individuals to work with us on the most interesting and challenging AI questions of our time. Our vision is bold and broad - to build systems that have true artificial intelligence across agents, applications, services, and infrastructure. It's also inclusive: we aim to make AI accessible to all - consumers, businesses, developers - so that everyone can realize its benefits.

Microsoft AI (MAI) is seeking experienced Member of Technical Staff - Copilot AI Evaluation Engineering Manager to lead building our LLM evaluation solutions for Copilot. You'll work in a deeply iterative, collaborative, fast-paced environment to improve our technology for millions of users around the world. You will lead a team of talented software engineers and applied AI engineers to deliver world-class evaluation solutions that are secure, reliable, and scalable, while collaborating closely with cross-functional teams including Product and Research.

We value engineers who are self-starters, care deeply about the end user experience, and take pride in building products to solve customer needs.

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Starting January 26, 2026, MAI employees are expected to work from a designated Microsoft office at least four days a week if they live within 50 miles (U.S.) or 25 miles (non-U.S., country-specific) of that location. This expectation is subject to local law and may vary by jurisdiction.

Responsibilities

* Hire, manage, and lead a team of software engineers, AI engineers, and machine learning engineers responsible for delivering world-class LLM evaluation solutions.

* Collaborate with Eng and Product leadership to prioritize features and improve our world-class AI companion to the world

* Design and build evaluation platforms, novel evaluation techniques, and agentic solutions for measuring and improving copilot quality.

* Drive implementation of features and systems, breaking down long-term goals into clear milestones, aligning with release plans, and ensuring cross-team coordination.

* Embody our Culture and Values.

Qualifications

Required Qualifications

* Bachelor's Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python

Preferred Qualifications

* Bachelor's Degree in Computer Science, or related technical discipline AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

* 3+ years people management experience.

* Hands on experience with ML and/or AI evaluation OR equivalent experience.

* Experience leading engineering teams to deliver large-scale software systems, preferably in AI, machine learning, graphics or related fields.

* Thrive in a fast-paced, collaborative environment and are comfortable making progress in ambiguity.

* Enjoy working closely with cross-functional partners and teammates in an inclusive, curious culture.

* Have strong opinions about best investments to make in establishing the most delightful and performant AI companion engineering system.

Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain…