AI Benchmarking Spec Italian, SFIX

The Seller AI team within International Seller Services focuses on helping sellers harness Gen-AI/LLM powered tools and agentic solutions that enable accelerated business growth on Amazon. Our primary focus is handling annotations for training, measuring, and improving Artificial Intelligence (AI) and Large Language Models (LLMs), enabling Amazon to deliver a superior seller experience worldwide. The AI Benchmarking Associate evaluates AI systems by designing and executing benchmarking and audit activities to assess model quality, compliance, robustness, and fairness.

This role blends AI auditing, quality assurance, and traditional audit documentation and stakeholder communication. By joining us, you will play a pivotal role in shaping the future of selling on Amazon for sellers worldwide.

• Assist in planning and executing benchmarking exercises for AI models, including defining test plans, metrics, and acceptance criteria across accuracy, robustness, bias, and reliability.
• Support content accuracy, relevancy, and privacy checks by reviewing datasets, model outputs, and data handling practices, escalating potential regulatory risks.
• Validate data based on specific annotation guidelines, ensuring the accuracy and quality of the collected information.
• Prepare clear audit and benchmarking reports, including error ratings, root‑cause analysis, and recommendations, and contribute to presentations for senior stakeholders.
• Maintain organized audit documentation, evidence, and benchmarking datasets to support internal review.
• Work closely with team members and managers to drive process efficiencies and explore opportunities for automation.
• Strive to enhance the productivity and effectiveness of data generation by contributing to the development and continuous improvement of AI audit methodologies, checklists, and test frameworks as regulations and best practices evolve.

We are open to hiring candidates to work out of the following location:
Bengaluru, Karnataka, IND.

• Speak, write, and read fluently in Italian Language.
• Min. B2.2 OR C1 certified in Italian Language.
• Bachelor's degree or equivalent with 3+ years of equivalent experience.

• Masters Degree or C1 certified in Italian Language.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit (Use the "Apply for this Job" box below). for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.