AI Language Model Task Evaluator - Remote

Overview

This role involves enhancing large-language-model performance on realistic reasoning tasks by creating and evaluating tasks and solutions.

In Short

• Write challenging, creative, and realistic tasks for AI models to solve
• Write gold-standard solutions to those tasks
• Evaluate AI-generated responses for clarity, accuracy, and comprehensiveness
• Collaborate directly with Mercor coordinators and client’s research leads
• Availability of up to 40 hours of work per week
• Fully remote and asynchronous work, flexible to your schedule
• Project length is a minimum of 6 weeks with potential extensions
• Legally classified as an hourly contractor for Mercor
• Payment at the end of each week via Stripe Connect
• Complete a short interview and participate in a paid work trial

Requirements

• Experience in technical writing, editing, research analysis, or related fields
• Demonstrable experience creating Q&A, summaries, or problem-solving tasks
• Ability to excel at deep reading and synthesize information
• Strong fact-checking and editing skills
• Analytical judgment to spot inconsistencies in draft outputs
• Clear and concise written communication with high attention to detail

Benefits

• Flexible work schedule
• Remote work opportunity
• Weekly payment
• Opportunity for project extensions
• Experience with top AI labs