Machine Learning Scientist​/Sr Scientist, Federated Benchmarking & Validation Engineering

Machine Learning Scientist/Sr Scientist, Federated Benchmarking & Validation Engineering

Purpose:

Lilly Tune Lab is an AI-powered drug discovery platform that provides biotech companies with access to machine learning models trained on Lilly's extensive proprietary pharmaceutical research data. Through federated learning, the platform enables Lilly to build models on broad, diverse datasets from across the biotech ecosystem while preserving partner data privacy and competitive advantages.

This role centers on constructing robust validation frameworks for federated models, creating privacy-preserving test sets across partner datasets, establishing standardized benchmarks against public datasets, and ensuring model reproducibility and generalization in diverse deployment scenarios.

• Federated Test Set Design:
  Architect and implement privacy-preserving protocols for constructing representative test sets across distributed partner datasets, ensuring statistical validity while maintaining data isolation.
• Benchmark Suite Development:
  Create comprehensive benchmark suites covering small molecules, antibodies, and RNA therapeutics.
• Cross-Domain Validation:
  Develop validation strategies that assess model generalization across different experimental protocols, cell lines, species, and therapeutic indications while respecting partner data boundaries.
• Public Dataset Integration:
  Systematically benchmark federated models against public datasets to establish performance baselines and identify gaps.
• Validation Frameworks:
  Implement proper scaffold-split validation protocols that assess model performance on prospective data.
• Reproducibility Infrastructure:
  Build robust MLOps pipelines ensuring complete reproducibility of federated experiments, including versioning of data snapshots, model checkpoints, and hyperparameter configurations.
• Statistical Rigor:
  Design statistically powered validation studies accounting for multiple testing, hierarchical data structures, and non-independent observations common in drug discovery datasets.
• Performance Profiling:
  Develop comprehensive performance profiling across diverse molecular scaffolds, target classes, and property ranges, identifying systematic biases and failure modes.
• Platform Integration:
  Collaborate with engineering teams to integrate validation frameworks with the Tune Lab federated learning platform built on NVIDIA FLARE.

• PhD in Computational Biology, Bioinformatics, Cheminformatics, Computer Science, Statistics, or related field from an accredited college or university.
• Minimum of 2 years of experience in the biopharmaceutical industry or related fields, with demonstrated expertise in drug discovery and early development.
• Strong foundation in experimental design, statistical validation, and hypothesis testing.
• Experience with ML model validation, cross-validation strategies, and performance metrics.
• Proficiency in data engineering, pipeline development, and automation.

• Experience with federated learning platforms and distributed computing.
• Knowledge of regulatory requirements for AI/ML in pharmaceutical development.
• Expertise in ADMET assay development and validation.
• Understanding of antibody engineering and characterization methods.
• Familiarity with RNA therapeutic design and delivery systems.
• Experience with clinical biomarker validation and translational research.
• Proficiency in workflow orchestration tools.
• Strong knowledge of containerization and cloud computing.
• Publications on model validation, benchmarking, or reproducibility.
• Experience with GxP compliance and quality management systems.
• Exceptional attention to detail and commitment to scientific rigor.
• Strong technical writing skills for regulatory documentation.

This role is based at a Lilly site in Indianapolis, South San Francisco, or Boston with up to 10% travel (attendance expected at key industry conferences). Relocation is provided.

Lilly is a proud EEO Employer and does not discriminate on the basis of age, race, color, religion, gender identity, sex, gender expression, sexual orientation, genetic information, ancestry, national origin, protected veteran status, disability, or any other legally protected status. Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form.

Anticipated wage: $151,500 - $244,200. Full-time employees also eligible for company bonus and comprehensive benefit program.