Tech Lead Manager, Machine Learning Research Scientist- LLM Evals job opportunity at Scale AI.



bot
Scale AI Tech Lead Manager, Machine Learning Research Scientist- LLM Evals
Requires: 5-years - XP
Pattern: full-time
apply Apply Now
Salary:
Status:
Copy Link Report
Other
San Francisco, United States Of America
San Francisco....United States Of America

Lead a team of highly effective #research scientists and research engineers on #LLM evals. Conduct research on the effectiveness and limitations of existing LLM evaluation techniques. Design and develop novel #evaluation benchmarks for large language models, covering areas such as instruction following, factuality, robustness, and fairness. #Communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects. Collaborate with internal teams and external partners to refine metrics and create standardized evaluation protocols. Implement scalable and reproducible evaluation pipelines using modern ML frameworks. Publish research findings in top-tier #AI conferences and contribute to open-source #benchmarking initiatives. Remain up-to-date on ongoing research in the team, help work through technical challenges, and be involved in design decisions Remain deeply involved in the research #community, both understanding trends, and setting them Thrive in a high-energy, fast-paced startup environment and are ready to #dedicate the time and effort needed to drive impactful results.

Other Ai Matches

Medical Fellow - Human Frontier Collective (US) Applicants are expected to have a solid experience in handling Researcher related tasks
Solutions Engineer, Enterprise Applicants are expected to have a solid experience in handling Engineer related tasks
Field Engineer, Public Sector Applicants are expected to have a solid experience in handling Engineering related tasks
Medical Fellow - Human Frontier Collective (US) Applicants are expected to have a solid experience in handling Medical related tasks
Senior Software Engineer, Enterprise GenAI Applicants are expected to have a solid experience in handling Engineering related tasks
Solutions Engineer - Robotics Applicants are expected to have a solid experience in handling Solutions Engineer related tasks
Product Manager, Gen AI Platform Applicants are expected to have a solid experience in handling Manager related tasks
Director of Corp Development and Investor Relations, Finance Applicants are expected to have a solid experience in handling Director related tasks
Applied AI Engineering Manager, Enterprise Applicants are expected to have a solid experience in handling Manager related tasks
Tech Lead Manager- MLRE, ML Systems Applicants are expected to have a solid experience in handling Lead Manager related tasks
Forward Deployed Engineering Manager, GenAI Applications Applicants are expected to have a solid experience in handling Deployed Engineer related tasks
Engagement Manager, International Public Sector Applicants are expected to have a solid experience in handling Manager related tasks
Operations Specialist Applicants are expected to have a solid experience in handling Operations Specialist related tasks
Machine Learning Research Engineer - Robotics Applicants are expected to have a solid experience in handling Engineering related tasks
Legal Fellow - Human Frontier Collective (US) Applicants are expected to have a solid experience in handling Legal related tasks
Director, Public Sector GTM Strategy Applicants are expected to have a solid experience in handling Director related tasks
Staff Software Engineer - Developer Experience Applicants are expected to have a solid experience in handling Software Engineer related tasks
Senior Software Engineer, Data Experience Applicants are expected to have a solid experience in handling Software Engineer related tasks
Machine Learning Research Scientist/ Engineer, Agents Applicants are expected to have a solid experience in handling Engineering related tasks
Software Engineering Intern (Summer 2026) Applicants are expected to have a solid experience in handling Software Engineer related tasks
Head of Evaluation and Oversight Research Applicants are expected to have a solid experience in handling Research related tasks
Visiting Faculty Applicants are expected to have a solid experience in handling Visiting Faculty related tasks
Instructional Designer Applicants are expected to have a solid experience in handling Designer related tasks