AI Engineer

At Robots & Pencils, we build meaningful, scalable digital products by blending strategy, design, and engineering. We are seeking a Level 4 AI Engineer to build production LLM applications for an enterprise client as part of a long‑term, delivery‑focused engagement.

You will own the AI stack end‑to‑end, including RAG pipelines, prompt engineering, and evaluation frameworks. This is a hands‑on role: you will write production code, tune prompts, build evaluation and observability systems, and iterate based on real user feedback.

There is a working proof of concept in place. Your responsibility is to make it production‑ready and extend it with intelligent, reliable features that operate at enterprise scale.

• Build, optimize, and evolve RAG pipelines, including retrieval strategies, chunking, and re‑ranking
• Develop prompts and guardrails for domain‑specific LLM applications
• Implement hallucination detection, mitigation, and fact‑checking mechanisms
• Build embeddings‑based search and recommendation features
• Validate AI features with real users and iterate based on qualitative and quantitative feedback

• Set up and maintain LLM evaluation frameworks to measure quality, relevance, and reliability
• Implement observability and monitoring for production AI systems
• Monitor live AI systems and resolve quality, accuracy, and performance issues
• Continuously improve AI outputs based on evaluation data and user behavior

• Work closely with product and engineering teams to integrate AI into user‑facing features
• Build and maintain backend services in Python
• Integrate with vector databases to support retrieval and semantic search workflows
• Ensure AI solutions meet enterprise requirements for security, scalability, and maintainability

• Collaborate with cross‑functional partners across product, engineering, and design
• Operate effectively in environments with evolving requirements and ambiguity
• Communicate clearly with technical and non‑technical stakeholders
• Take ownership of delivery outcomes from experimentation through production

• 8+ years of professional software engineering experience, with 4+ years focused on applied AI/ML or data‑driven systems in production environments
• 3+ years building and operating production AI systems
• Strong hands‑on experience with LLM applications, including RAG, prompt engineering, and evaluation
• Experience implementing hallucination detection and mitigation techniques
• Proficiency in Python
• Experience working with vector databases (Weaviate, Pinecone, or similar)
• Experience with LLM evaluation frameworks (Langfuse, Weights & Biases, or custom solutions)
• Production experience using Claude and/or GPT APIs
• Strong understanding of embeddings and semantic search
• Comfortable working with ambiguity and iterating on unclear problems
• Bachelor's degree in computer science, engineering, data science, or a related technical field, or equivalent practical experience
• Advanced degree (Master’s or PhD) in a relevant field

• Experience with Azure AI services, including Azure OpenAI and Cognitive Services
• Experience with document processing (PDF extraction, OCR)
• Exposure to audio or speech processing (e.g., Whisper or similar tools)
• Experience building enterprise B2B software
• Experience with ML classification and model training

• LLMs:
  Claude (Anthropic), Azure OpenAI
• Vector Database:
  Weaviate
• Backend:
  Python
• Infrastructure:
  Azure
• Evaluation & Observability:
  Langfuse or similar

• You are hands‑on and delivery‑focused, writing code and owning outcomes
• You balance speed with quality in production environments
• You communicate clearly and collaborate effectively across disciplines
• You take ownership of ambiguous problems and drive them to resolution
• You prioritize reliability, maintainability, and real‑world impact

• Real production impact not a POC that sits on a shelf
• Exposure to the full AI lifecycle: RAG, LLM applications, evaluation, classification, and monitoring
• End‑to‑end ownership of the AI stack and technical decision‑making
• A small, senior team with direct access to enterprise clients