Job Description & How to Apply Below
My Remote Team , Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.
Job Title:
AI Engineer
Experience:
5+ Years
Working Model: Remote
Responsibilities
Build Python-based pipelines for automated quality testing of AI responses.
Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
Automate regression and stress testing for conversational AI flows.
Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
Implement both rule-based and AI-driven quality checks.
Monitor model drift, bias, and hallucinations using automated workflows.
Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
Develop monitoring dashboards to visualize conversation quality.
Collaborate with ML engineers, product managers, and QA teams to close the feedback loop.
Experiment with prompt engineering and automated prompt-testing frameworks.
Explore reinforcement learning, self-critique models, or human-in-the-loop automation for continuous improvement.
Automate compliance and policy adherence checks for enterprise AI systems.
Requirements
Strong 5 Years of experience in Python development (automation, scripting, data handling).
Experience with LLMs/NLP frameworks.
Understanding of MLOps / AI deployment pipelines.
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×