Customer Service & Support AI Rater & Evaluator - Remote

Overview

LILT is building a global network of domain experts to support high-quality AI evaluation across training, benchmarking, red-teaming, and ongoing model monitoring. We are seeking customer service and support professionals to contribute expert judgment to human-in-the-loop AI evaluation workflows used by leading enterprises and hyperscalers.

This role is designed for professionals who understand how customer support interactions work in real operational environments and who can apply that expertise to evaluate, assess, and improve multilingual AI systems used in customer-facing contexts.

Your contribution of expertise will directly influence multilingual AI model quality, safety, and deployment readiness.

This role includes two distinct expert tracks, based on experience level and scope of responsibility.

In Short

• Evaluate AI outputs related to customer service and support interactions
• Perform structured scoring, comparison, classification, and judgment tasks
• Assess accuracy, clarity, tone, helpfulness, and alignment with support best practices
• Identify hallucinations, misleading responses, policy violations, or unsafe guidance
• Apply domain-specific customer support guidelines consistently across tasks
• Validate and refine evaluation rubrics and edge-case handling
• Conduct error analysis and qualitative reviews of model behavior
• Partner with LILT research, product, and customer teams on evaluation design
• Support red-teaming, policy alignment, and model readiness assessments
• Engagement model is contract-based with flexible participation

Requirements

• Customer support professionals, service operations specialists, or CX practitioners
• Experience handling customer inquiries, support workflows, or service escalation
• Strong attention to detail and comfort working with structured evaluation criteria
• Deep domain expertise in customer service, support operations, or CX
• Ability to clearly explain nuanced service decisions and tradeoffs
• Reliability, professionalism, and respect for quality standards
• Native or professional fluency in one or more supported languages
• English fluency required for guidelines, feedback, and collaboration

Benefits

• Earn money while contributing to AI evaluation
• Work on diverse projects from anywhere, any time you want
• Get paid quickly and fairly
• Build your professional network in a supportive community
• Engagement model offers opportunities for recurring work based on performance