AI Tester Automation Testing

About the role

As an AI Tester with Automation expertise, you will make an impact by validating the quality, performance, and reliability of next generation AI and LLM powered applications. You will be a valued member of the Quality Engineering and Assurance team and work collaboratively with AI engineers, automation architects, and cross functional stakeholders to ensure robust, scalable, and compliant AI solutions.

• Design and execute automated test scripts using Python and Playwright for AI, LLM, and ML workloads.
• Validate LLM based applications, including prompt flows, RAG pipelines, vector search, and retrieval logic.
• Test multi agent orchestration workflows, including MCP/A2A protocol–based interactions.
• Conduct functional, integration, API, and data validation testing across complex ML pipelines.
• Collaborate with engineering teams to troubleshoot defects, optimize model behavior, and ensure compliance and accuracy of AI outputs.

At Cognizant, we strive to provide flexibility wherever possible, and we are here to support a healthy work life balance through our various wellbeing programs. Based on this role’s business requirements, this is an onsite position requiring presence in Weehawken, New Jersey. The working arrangements for this role are accurate as of the date of posting and may change based on project engagement, business needs, or client requirements.

• 5 to 7 yrs. of QA experience with
• Strong programming proficiency in Python and hands on automation experience with Playwright.
• Working knowledge of AI/ML algorithms, including NLP, classification, clustering, and LLM evaluation.
• Experience with LLM development frameworks such as Lang Chain, Lang Graph, vector databases, RAG architectures, and prompt engineering.
• Familiarity with multi agent AI architectures, including multi agent orchestration using MCP and A2A protocols.
• Strong analytical skills with a solid understanding of data modeling and data transformation using SQL.
• Experience testing or consuming APIs, RESTful services, and data connectors.
• Awareness of source code control systems such as Git Lab, along with CI/CD build and deployment pipelines.

• Hands on experience with Generative AI, GPT models, or enterprise LLM deployments.
• Exposure to cloud platforms—especially Microsoft Azure—for AI, ML, or infrastructure workloads.
• Good understanding of Neo4j, knowledge graphs, or graph based reasoning for LLM workflows.
• Experience testing end to end AI pipelines, including data ingestion, evaluation, and observability.
• Ability to work in fast paced, innovative environments delivering scalable AI solutions

Cognizant will only consider applicants for this position who are legally authorized to work in the United States without company sponsorship. Please note, this role is not able to offer visa transfer or sponsorship now or in the future. We're excited to meet people who share our mission and can make an impact in a variety of ways. Don't hesitate to apply, even if you only meet the minimum requirements listed.

Think about your transferable experiences and unique skills that make you stand out as someone who can bring new and exciting things to this role.

Applications will be accepted until 4/17/2026.

The annual salary for this position is between $53,477 and $92,500 depending on experience and other qualifications of the successful candidate.

This position is also eligible for Cognizant’s discretionary annual incentive program, based on performance and subject to the terms of Cognizant’s applicable plans.

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability
• Paid Parental Leave
• Employee Stock Purchase Plan