Senior Principal MLOps Engineer, AI Inference

Senior Principal MLOps Engineer, AI Inference

At Red Hat we believe the future of AI is open, and we are on a mission to bring the power of open‑source large language models and vLLM to every enterprise. Red Hat Inference accelerates AI for the enterprise and brings operational simplicity to GenAI deployments. As leading developers, maintainers of the vLLM project, and inventors of state‑of‑the‑art techniques for model compression, our team provides a stable platform for enterprises to build, optimize, and scale LLM deployments.

We are seeking an experienced MLOps engineer to work closely with our product and research teams to scale state‑of‑the‑art deep learning products and software. In this role, you will manage training and deployment pipelines, build and release the Red Hat AI Inference runtimes, continuously improve Dev Ops and CI/CD tooling, and find opportunities to automate procedures and tasks.

• Collaborate with research and product development teams to scale machine learning products for internal and external applications.
• Create and manage model training and deployment pipelines.
• Actively contribute to managing and releasing upstream and midstream product builds.
• Test to ensure correctness, responsiveness, and efficiency.
• Troubleshoot, debug, and upgrade Dev & Test pipelines.
• Identify and deploy cybersecurity measures by performing vulnerability assessments and risk management.
• Collaborate with a cross‑functional team about market requirements and best practices.
• Keep abreast of the latest technologies and standards in the field.

• 2+ years of experience in MLOps, Dev Ops, Automation and modern software deployment practices.
• Strong experience with Git, Git Hub Actions including self‑hosted runners, Terraform, Jenkins, Ansible, and common automation/monitoring technologies.
• Highly experienced with administering Kubernetes/Open Shift.
• Familiarity with Agile development methodology.
• Experience with cloud computing on at least one of AWS, GCP, Azure, or IBM Cloud.
• Solid programming skills especially in Python.
• Solid troubleshooting skills.
• Ability to interact comfortably with a distributed team.
• Experience maintaining infrastructure and ensuring stability.
• Familiarity with contributing to the vLLM CI community is a plus.
• While a bachelor’s degree or higher in computer science, mathematics, or a related field is valued, we prioritize technical prowess, initiative, problem solving, and practical experience.

Salary range: $ - $. Actual offer will be based on your qualifications.

• Comprehensive medical, dental, and vision coverage.
• Flexible Spending Account – healthcare and dependent care.
• Health Savings Account – high deductible medical plan.
• Retirement 401(k) with employer match.
• Paid time off and holidays.
• Paid parental leave plans for all new parents.
• Leave benefits including disability, paid family medical leave, and paid military leave.
• Additional benefits: employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more.

Red Hat is proud to be an equal‑opportunity workplace and an affirmative action employer. We review applications for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law. Red Hat supports individuals with disabilities and provides reasonable accommodations to job applicants.

If you need assistance completing our online job application, email appl

References and additional details about the hiring process, location, and remote options are available on our careers site.