Software Development Manager, Neuron Tools, Annapurna Labs

Overview

AWS Neuron is the complete software stack for the AWS Inferentia and Trainium cloud-scale machine learning accelerators and the Trn1 and Inf1 servers that use them.

As the Software Development Manager for the Tools Team, you will be responsible for leading a talented team of engineers to develop and maintain high-performance monitoring and profiling tools for machine learning applications and AI accelerators. You will oversee the design, development, and deployment of the Neuron Profiler and other Neuron Tools. The profiler helps internal and external customers optimize AI workloads across hardware platforms such as Trainium and Inferentia devices by providing deep insights into performance bottlenecks and system behavior.

You will manage the full development lifecycle of the Neuron Profiler/Tools toolchain, ensuring scalability, reliability, and usability. You will collaborate with cross-functional teams to ensure that our C++ compiler and runtime generate key information so customers can understand and optimize the performance of our custom hardware. Additionally, you will drive innovations that allow the profiler to support multiple frameworks, such as PyTorch, Tensor Flow, and XLA.

A successful candidate will have an established background in building AI/ML and performance analysis tools. Experience with ML-specific profiler tools (like PyTorch Profiler or Tensor Flow Profiler) is highly desirable, along with direct customer-facing experience and a strong motivation to achieve results.

• Lead a team of engineers to develop and maintain high-performance monitoring and profiling tools for ML applications and AI accelerators, including the Neuron Profiler and Neuron Tools.

• Manage the full development lifecycle of the profiler/toolchain, focusing on scalability, reliability, and usability.

• Collaborate with cross-functional teams to ensure the C++ compiler and runtime provide essential performance information to customers.

• Drive innovations to support multiple frameworks (e.g., PyTorch, Tensor Flow, XLA).

• Engage with customers and internal teams to understand needs and translate them into product directions and deliverables.

• Basic Qualifications

  • 3+ years of engineering team management experience

  • 7+ years of working directly within engineering teams

  • 3+ years of designing or architecting (design patterns, reliability and scaling) of new and existing systems

  • Experience partnering with product or program management teams

  • Experience in C++, Go, and Python

• Preferred Qualifications

  • 2+ years of experience leading teams in Machine Learning development, including building and training large models, working with PyTorch and/or Tensor Flow using large distributed fleets of GPUs or other accelerators

  • Experience with Linux distributions such as Ubuntu or CentOS, kernel development, and tooling such as perf and gdb

  • Experience with performance profiling, tracing, and analysis of AI training/inference applications

  • Experience with large-scale distributed AI training/inference applications, including libfabric, MPI, Slurm, and EKS

  • Experience with fleet monitoring, debugging, and reliability

  • Knowledge of AI-powered optimization suggestions for profiling is advantageous

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Los Angeles County applicants:
Job duties include safe and cooperative work, adherence to standards of excellence under stress, effective communication, and compliance with laws and company policies. Criminal history may affect eligibility. We will consider qualified applicants with arrest and conviction records per the Los Angeles County Fair Chance Ordinance.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation during the application and hiring process, please visit (Use the "Apply for this Job" box below). for more information.

The base salary range is listed below. Your Amazon package includes sign-on payments and RSUs. Final compensation is based on experience, qualifications, and location. Benefits include health insurance, 401(k) matching, paid time off, parental leave, and more. Learn more at .

USA, CA, Cupertino -  -  USD annually

USA, WA, Seattle -  -  USD annually