Senior Product Manager – DPAI Arena

Amsterdam, Netherlands;
Belgrade, Serbia;
Berlin, Germany;
Limassol, Cyprus;
Madrid, Spain;
Munich, Germany;
Paphos, Cyprus;
Prague, Czech Republic;
Tel Aviv, Israel;
Warsaw, Poland;
Yerevan, Armenia

About us
At Jet Brains, code is our passion. Ever since we started, we have strived to make the strongest, most effective developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create.

We believe in building tools developers love – and we see the next frontier in AI-powered developer tools. The DPAI Arena project aims to define and maintain an open, community-driven benchmark for evaluating AI agents and IDE-embedded AI features  believe that the benchmark industry is only in its early stages, and we’re planning to take part in the evolution of the field.

We’re looking for a Senior Product Manager to own the direction of DPAI Arena and help define what “good evaluation” looks like and how we measure it.

• Refine our answer to the question of what makes an evaluation task a representative data point for software engineers and IDE AI features
  , using usage data, feedback loops, and community input.
• Define and iterate the evaluation methodology, designing tasks, specifying metrics and success criteria, and covering real developer workflows from both product and UX perspectives.
• Build and maintain relationships with external vendors, platform partners, outsourcing providers, and internal stakeholders to expand participation in DPAI Arena and drive both business and ecosystem development.
• Curate and manage datasets, coordinating contributions from Jet Brains teams, external partners, and outsourcing vendors (including crowdsourced workers when needed) to keep evaluation tasks high-quality, representative, and diverse.
• Own the evaluation standard and roadmap, representing DPAI Arena internally and externally while aligning plans with the broader Jet Brains AI and IDE strategy.

• Proven experience in product management (developer tools or data-driven products is a plus).
• Experience working on GenAI features for developer tooling.
• Strong analytical skills, with the ability to define meaningful metrics, interpret results, and turn them into product design.
• Experience working with multiple stakeholders: internal teams, external partners, and vendors (including outsourcing providers).
• Understanding of developer workflows, IDEs, and AI-assisted development, and the ability to translate real-world coding tasks into evaluation workloads.
• Strong communication skills, both written and spoken.
• Ownership mindset and proactive approach.
• Engineering background (particularly SWE or QA for tech products).
• Experience maintaining or curating open-source products.
• ML experience.
• Familiarity with outsourcing or crowd sourcing platforms, data collection pipelines, or community-driven ecosystems.

Interested in building your career at Jet Brains? Get future opportunities sent straight to your email.

By checking this box, I agree to allow Jet Brains to retain my data for future opportunities for employment for up to 2190 days after the conclusion of consideration of my current application for employment.