AIML - ML Engineer, Safety Human Evaluation

Job Summary
:
Apple is seeking Machine Learning Engineers to work on the next generation of generative AI applications. The role focuses on assessing and improving the safety and inclusivity of Apple's Generative-AI powered features, requiring innovative problem-solving in machine learning, particularly with large language models and multimodal applications.

Responsibilities
:

• Develop metrics for evaluation of safety and fairness risks inherent to generative models and Gen-AI features

• Design datasets, identify data needs, and work on creative solutions, scaling and expanding data coverage through human and synthetic generation methods

• Develop sampling strategies and combine human annotation with auto-grading to deliver high-quality, high-confidence insights at a fast pace and large scale

• Use and implement data pipelines, and collaborate cross-functionally to execute end-to-end safety evaluations

• Distill project findings into recommendations for product engineering teams and safety policy development

• Develop ML-based enhancements to red teaming, model evaluation, and other processes to improve the quality of Apple Intelligence’s user-facing products

• Work with highly-sensitive content with exposure to offensive and controversial content

Qualifications
:
Required
:

• MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field with a strong basis in scientific data collection and analysis + at least 4 years of relevant work experience, or BA/BS with 8+ years of relevant work experience

• Experience gathering and analyzing language data, image data, and/or multi-modal data, including LLM-generated data

• Strong experience designing human annotation projects, writing guidelines, and dealing with highly multi-labeled, nuanced, and often conflicting data

• Proficiency in data science, machine learning, analytics, and programming with Python & Pandas; strong experience with one or more plotting & visualization libraries

• Ability to collaborate with team members to prioritize competing projects, set and maintain a schedule for milestones and project completions, and communicate with all levels of team members as well as external stakeholders

• Strong skills for rigorous model quality metrics development; interpretation of experiments and evaluations; and presentation to executives

Preferred
:

• Experience working in the Responsible AI space

• Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable

• Prior scientific research and publication experience

• Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures

• Proven track record of contributing to diverse teams in a collaborative environment

• A passion for building outstanding and innovative products. This position involves a wide variety of interdisciplinary skills

Company
:
Apple is a technology company that designs, manufactures, and markets consumer electronics, personal computers, and software. Founded in 1976, the company is headquartered in Cupertino, USA, with a team of 10001+ employees. The company is currently Late Stage.