AI Quality Assurance

Location: Bengaluru

Responsibilities
Own end-to-end testing strategy for AI-driven web applications and backend services
Design and maintain automated test frameworks for scalable AI systems
Execute smoke, functional, regression, integration, and performance testing
Evaluate LLM/model performance using structured evaluation frameworks
Conduct backend and REST API testing using tools like Swagger and Postman
Perform load and performance testing (e.g., JMeter)
Identify, document, and track defects through resolution
Contribute to CI/CD pipelines to strengthen automated testing workflows
Validate deployments in Kubernetes and AWS environments

Requirements
Must-Have
5–7 years of experience in software testing with strong automation expertise
Hands-on experience testing AI-based applications or LLM-powered systems
Experience in model evaluation and analyzing LLM performance
Strong backend/API testing experience
Proficiency in Python, JavaScript, or Type Script

Experience with automation tools and frameworks
Working knowledge of Kubernetes (log inspection/debugging)
Experience using Git Hub and JIRA in Agile environments
Strong Plus

Experience with RAGA’S, Deep Eval, or similar LLM evaluation frameworks

Experience with pytest or modern automation tooling
Exposure to open-source AI/ML models
Experience contributing to CI/CD pipelines (e.g., Git Hub Actions)
Experience testing applications deployed across multiple geographies
Hands-on AWS experience