Tech Lead Manager, Machine Learning Research Scientist- LLM Evals job opportunity at Scale AI.

Other

San Francisco, United States Of America

San Francisco....United States Of America

summarize with Ai

Lead a team of highly effective #research scientists and research engineers on #LLM evals. Conduct research on the effectiveness and limitations of existing LLM evaluation techniques. Design and develop novel #evaluation benchmarks for large language models, covering areas such as instruction following, factuality, robustness, and fairness. #Communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects. Collaborate with internal teams and external partners to refine metrics and create standardized evaluation protocols. Implement scalable and reproducible evaluation pipelines using modern ML frameworks. Publish research findings in top-tier #AI conferences and contribute to open-source #benchmarking initiatives. Remain up-to-date on ongoing research in the team, help work through technical challenges, and be involved in design decisions Remain deeply involved in the research #community, both understanding trends, and setting them Thrive in a high-energy, fast-paced startup environment and are ready to #dedicate the time and effort needed to drive impactful results.