Applied Research Scientist, LLM Evaluation & Post-Training - Remote

Overview

As an Applied Research Scientist, you will lead research and experimentation on LLM evaluation and post-training methods, working closely with engineers and data scientists to drive model improvement.

In Short

• Lead research on LLM evaluation and post-training.
• Design and validate evaluation frameworks.
• Collaborate with AI/ML engineers and data scientists.
• Conduct rigorous experiments and statistical analysis.
• Develop benchmark datasets and evaluation protocols.
• Engage with customers for technical recommendations.
• Produce high-quality documentation and reports.
• Contribute to thought leadership in LLM evaluation.
• Work in a highly collaborative environment.
• Preferred PhD in relevant field.

Requirements

• 5+ years of experience in applied ML research or related field.
• Strong coding skills in Python.
• Experience with ML frameworks like PyTorch or TensorFlow.
• Ability to design reproducible evaluation studies.
• Strong communication and collaboration skills.
• Experience with LLM evaluation and benchmarking.
• Expertise in experimental design and statistical analysis.
• Understanding of evaluation methodologies and metrics.
• Familiarity with human evaluation methods.
• Experience in customer-facing research roles is a plus.

Benefits

• Opportunity to work on cutting-edge AI technologies.
• Collaborative and innovative work environment.
• Access to a global workforce and diverse projects.
• Professional development opportunities.
• Competitive salary and benefits package.