QA Engineer

Founded in 2012, H2O.ai is on a mission to democratize AI. As the world’s leading agentic AI company, H2O.ai converges Generative and Predictive AI to help enterprises and public sector agencies develop purpose-built GenAI applications on their private data. With a focus on Sovereign AI—secure, compliant, and infrastructure‑flexible deployments—H2O.ai delivers solutions that align with the highest standards of data privacy and control.

Our open‑source technology has been adopted by over 20,000 organizations worldwide, including more than half of the Fortune 500. H2O.ai powers AI transformation for companies such as AT&T, Commonwealth Bank of Australia, Chipotle, Workday, Progressive Insurance, and NIH.

You will be a core QA engineer responsible for the end‑to‑end quality and reliability of H2O.ai's product portfolio. This is a hybrid position based in Mountain View, CA.

• Design, build, and maintain Python‑first automation frameworks (pytest + Playwright + async) for UI, API, and end‑to‑end workflows of H2O.ai's product portfolio.
• Perform deep exploratory and manual testing of the H2O.ai's product portfolio web UI (chat interfaces, document upload/processing, agent builder, maker suite, evaluation hub, enterprise admin console).
• Use H2O.ai's product portfolio itself (and other GenAI tools) in creative ways to:
  • Generate realistic test documents, datasets, and edge‑case prompts.
  • Auto‑generate or refine test cases via prompt engineering.
  • Rapidly summarize and debug massive, complex log files (e.g., Kubernetes pods, etc.).
  • Explain cryptic LLM traceback chains or hallucination root causes in seconds instead of hours.
• Root‑cause of difficult, intermittent failures in distributed RAG/LLM systems by combining traditional log analysis with GenAI‑assisted debugging.
• Create and execute chaos experiments targeting LLM routing, vector database latency, GPU OOM, retrieval failures, and token‑limit edge cases.
• Build and manage ephemeral H2O.ai's product portfolio clusters on Kubernetes for testing (Helm, custom operators).
• Own UI regression suites (Playwright) and accessibility testing.
• Write reproducible, high‑quality bug reports that developers love and regularly verify fixes across the full stack.
• Collaborate closely with the H2O.ai's product portfolio feature teams in an Agile environment and influence testability from the design phase.

• 2‑4 years of QA experience with a strong mix of automation and hands‑on manual/exploratory testing.
• Decent Python skills and experience building maintainable test frameworks from scratch.
• Real‑world experience testing modern React/Type Script web applications and writing bulletproof Playwright or Selenium tests.
• Hands‑on Kubernetes experience in a testing or test‑environment context (kubectl, Helm, writing manifests, debugging pods).
• Proven ability to use generative AI tools daily to accelerate debugging, test‑data creation, and log analysis (you’ve already used H2O.ai's product portfolio, ChatGPT, Claude, or similar in your current QA workflow).
• Comfort reading and triaging complex logs from LLM frameworks, vector DBs, and tracing systems.
• Solid grasp of CI/CD (Git Hub Actions preferred) and infrastructure‑as‑code concepts.

• Prior experience testing RAG systems, agentic workflows, or enterprise chat/assistant platforms.
• Experience with visual diffing of generated outputs (documents, charts, markdown).
• Chaos engineering on Kubernetes (Chaos Mesh, Litmus) or GPU workloads.
• Familiarity with Chaos engineering principles.
• Basic understanding of containerization (Docker/Kubernetes concepts like pods and kubectl) in a testing context.

• Market leader in total rewards
• Remote‑friendly culture
• Flexible working environment
• Be part of a world‑class team
• Career growth
• Base salary range: $105, USD

H2O.ai is committed to creating a diverse and inclusive culture. All qualified applicants will receive consideration for employment without regard to their race, ethnicity, religion, gender, sexual orientation, age, disability status or any other legally protected basis.