Analyst - LLM/Prompt Evaluation (3 Month Fellowship) - Remote

Overview

This is a 3 month fellowship position with the potential for conversion to a full-time role based on performance and business needs. We are looking for an analyst to play a vital role in maintaining and improving the effectiveness, accuracy, and fairness of our Large Language Model-powered tools and analyses.

In Short

• Own the evaluation lifecycle for our LLM applications
• Conduct rigorous quality control and analysis
• Proactively enhance LLM safety and fairness
• Improve LLM effectiveness through iteration
• Document and communicate findings
• Collaborate with a diverse team of engineers and analysts
• Contribute to innovative work that evolves week by week
• Work remotely with potential for in-person collaboration
• Engage with the wider progressive political ecosystem
• Participate in a warm and supportive team environment

Requirements

• Experience in an analytical role involving data analysis or quality assurance
• Proficiency in SQL and basic Python scripting
• Exceptional attention to detail and methodical approach
• Strong analytical and problem-solving skills
• Motivated by high standards of quality and accuracy
• Interest in Large Language Models and prompt engineering
• Strong oral and written communication skills
• Team player who contributes to a positive work environment
• Eager to understand real-world impacts of work
• Willingness to engage with progressive campaigns and organizations

Benefits

• Competitive stipend for the fellowship period
• Medical, dental, and health benefits
• Supportive work environment
• Opportunity for conversion to full-time role
• Remote work flexibility