Remote PhD Rater - AI Trainer

1. Role Overview

Mercor is seeking experienced researchers and technical experts to contribute to a project supporting a frontier-model evaluation effort focused on agentic workflows. You’ll design and validate challenging benchmark tasks in data science, machine learning, finance, and coding to help surface and diagnose reasoning and problem-solving gaps in a target STEM model. The work centers on building robust, real-world tasks with executable tests and then analyzing model / agent behavior.

• Design challenging, real-world STEM problems
• Implement each task inside an agentic development environment using Python

• Deep expertise in data science, machine learning, finance, and / or Python-based coding
• Active or recently graduated PhD (Top 20 U.S.
  
  -based school)
• Strong research background in frontier STEM topics
• Ability to engage reliably for 30+ hours / week, primarily on weekdays
• Demonstrated technical output such as high-quality open-source contributions (especially in agentic / LLM tooling ecosystems)
• Comfort reading and reasoning about agent behavior traces to diagnose failure modes beyond surface-level errors

• Initial focus area: agentic workflows for STEM tasks
• Familiarity with agentic frameworks and OSS ecosystems is helpful (e.g., Lang Chain, MetaGPT, Auto Gen, AutoGPT, CrewAI, Llama Index, BabyAGI, Super
  
  AGI, CAMEL, Agent
  
  GPT, Dify, etc.)
• Deliverables are expected to be reproducible and testable (clear specs, deterministic tests where possible, documented environments)

• Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations.
• Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey.
• Thousands of professionals across domains like law, creatives, engineering, and research have joined Mercor to work on frontier projects shaping the next era of AI.