Evaluation Scenario Writer - AI Agent Testing Specialist - Remote

Overview

This role involves designing realistic evaluation scenarios for AI agents, ensuring clarity and reusability in testing procedures.

In Short

• Design structured test scenarios based on real-world tasks.
• Define acceptable agent behavior and golden paths.
• Annotate task steps and expected outputs.
• Collaborate with developers to test scenarios.
• Review agent outputs and adapt tests as needed.
• Work on projects aligned with personal skills.
• Shape the future of AI through innovative testing.
• Flexible scheduling for project contributions.
• Focus on ethical AI development.
• Utilize collective intelligence in AI projects.

Requirements

• Strong analytical skills.
• Attention to detail.
• Interest in AI decision-making processes.
• Experience in designing evaluation scenarios.
• Ability to work independently.
• Familiarity with AI and LLMs is a plus.
• Strong communication skills.
• Ability to adapt tests based on outputs.
• Experience in collaborating with technical teams.
• Willingness to learn and grow in the AI field.

Benefits

• Opportunity to work with leading tech innovators.
• Contribute to meaningful AI projects.
• Flexible work schedule.
• Engage with a community of AI specialists.
• Ethical focus on AI development.
• Potential for long-term collaboration.
• Enhance your skills in AI evaluation.
• Work remotely with a global team.
• Impact the future of AI technologies.
• Competitive compensation based on experience.