Legal & Compliance AI Rater/Evaluator - Japanese - Remote

Overview

LILT is building a global network of domain experts to support high-quality AI evaluation across training, benchmarking, red-teaming, and ongoing model optimization. We are seeking legal and compliance professionals to contribute expert judgment to human-in-the-loop AI evaluation workflows used by leading enterprises and governments.

This role is designed for professionals who understand how legal and regulatory language is used in real-world contexts and who can apply that expertise to evaluate, assess, and improve multilingual AI systems.

Your contribution of expertise will directly influence our multilingual AI model’s quality, safety, and deployment readiness.

This role includes two distinct expert tracks, based on experience level and scope of responsibility.

In Short

• Evaluate AI outputs related to legal and compliance content
• Perform structured scoring, comparison, classification, and judgment tasks
• Assess accuracy, relevance, completeness, risk, and alignment with legal norms
• Identify hallucinations, ambiguity, unsafe guidance, or misleading outputs
• Apply domain-specific guidelines consistently across tasks
• Validate and refine evaluation rubrics and edge-case handling
• Conduct error analysis and qualitative reviews of model behavior
• Partner with LILT research, product, and customer teams on evaluation design
• Support red-teaming, risk analysis, and model readiness assessments
• Comfort explaining nuanced legal reasoning clearly and precisely

Requirements

• Legal practitioner, compliance professional, legal analyst, or policy specialist
• Experience interpreting statutes, regulations, contracts, policies, or legal guidance
• Strong attention to detail and comfort working with structured evaluation criteria
• Deep domain expertise in law, compliance, or regulatory environments
• Strong judgment and ability to apply criteria consistently
• Reliability, professionalism, and respect for quality standards

Benefits

• Contract-based, flexible participation
• Project-based work with clear expectations and timelines
• Opportunities for recurring work based on performance and demand
• Compensation communicated upfront per project or task type