Machine Learning Scientist - Open Source Lead - Remote

Overview

LMArena is looking for a Machine Learning Scientist to lead our open-source research, including open data set and code releases, advancing how the world evaluates and understands AI models in the open.

In Short

• Lead open-source research in AI evaluation and methodology.
• Design and run experiments to evaluate AI model behavior.
• Collaborate with engineers and product teams to align research with user needs.
• Communicate findings through academic papers and conference talks.
• Analyze large-scale human voting and interaction data.
• Develop new metrics and evaluation protocols.
• Engage with the research community to foster collaboration.
• Prototype and test research ideas rapidly.
• Contribute to the scientific integrity of the LMArena leaderboard.
• Work with model providers to support responsible model testing.

Requirements

• PhD or equivalent experience in ML, NLP, or Statistics.
• Strong understanding of LLMs and deep learning architectures.
• Proficiency in Python and ML libraries like PyTorch, JAX, or TensorFlow.
• Experience designing and analyzing experiments with statistical rigor.
• Ability to translate research questions into practical systems.
• Passion for open science and community-driven research.
• Experience publishing research or working on open-source projects.
• Comfortable working with real-world usage data.
• Demonstrated ability to collaborate across teams.
• Strong communication skills for outreach and engagement.

Benefits

• Competitive salary and meaningful equity.
• Comprehensive healthcare coverage (medical, dental, vision).
• Work on cutting-edge AI with a mission-driven team.
• A culture that values transparency and community impact.
• Opportunity to shape the future of AI.