Evaluation Scenario Writer - AI Agent Testing Specialist - Remote

Overview

This role involves designing realistic and structured evaluation scenarios for AI agents, ensuring clarity and effectiveness in testing.

In Short

• Design structured test scenarios based on real-world tasks.
• Define the golden path and acceptable agent behavior.
• Annotate task steps, expected outputs, and edge cases.
• Collaborate with developers to test scenarios.
• Review agent outputs and adapt tests accordingly.
• Work on projects aligned with your skills and schedule.
• Contribute to shaping the future of AI.
• Ensure technology benefits everyone.
• Submit your resume in English.
• Interest in AI decision-making is beneficial.

Requirements

• Experience in designing evaluation scenarios.
• Strong analytical skills and attention to detail.
• Ability to create clear and structured documentation.
• Familiarity with AI and LLM-based technologies.
• Good communication skills.
• Ability to work independently and manage time effectively.
• Experience in testing and quality assurance is a plus.
• Knowledge of task simulation techniques.
• Proficiency in English.
• Interest in AI ethics and technology.

Benefits

• Flexible working hours.
• Opportunity to work on innovative AI projects.
• Contribute to meaningful technology initiatives.
• Collaborate with experts in the field.
• Enhance your skills in AI and evaluation design.
• Work in a supportive and innovative environment.
• Potential for future opportunities within the company.
• Engage in a mission-driven organization.
• Access to cutting-edge AI technology.
• Be part of a team shaping the future of AI.