Back-end Engineer

A fast-growing open-source platform is building a cutting‑edge feedback loop for optimizing large language model (LLM) applications turning production data into smarter, faster, and more cost‑effective models.

• Real-time feedback and metric collection
• Tools to optimize prompts, models, and inference strategies
• Built‑in A/B testing and experimentation frameworks
• Seamless observability and optimization pipelines

Our platform creates a continuous learning flywheel for LLMs, combining:

• Inference:
  One API for all LLMs with near‑zero overhead
• Observability:
  Feed inference data and feedback directly into your database
• Optimization:
  Improve from prompt engineering all the way to fine‑tuning and reinforcement learning
• Experimentation:
  Native support for routing, fallbacks, and advanced testing setups

We began building quietly in early 2024, ran a successful technical pilot by mid‑year, and launched our open‑source project in the fall. Backed by world‑class investors with experience supporting industry‑defining AI and database platforms, we have the resources and runway to pursue an ambitious, long‑term vision.

Were hiring a Founding Member of Technical Staff with deep systems engineering experience (especially in Rust).

Why not just engineer? Because we want cross‑functional builders people who can bridge research and production, infrastructure and experimentation. Our core systems are written in Rust, and the scope of work includes features like inference‑time optimizations and advanced experimentation frameworks.

• The chance to have most of your work be open‑source and highly visible
• Rapid personal growth across both systems and machine learning domains
• A collaborative, technical, in‑person environment where craftsmanship and ambition meet
• Backing from leading venture funds with experience in AI, infrastructure, and developer tools

• Strong technical foundation and the ability to drive large projects from start to finish
• Background in back‑end systems and infrastructure (especially with Rust)
• Hunger for personal and professional growth there are no limits here
• Willingness to work on‑site in NYC with a small, highly focused team