Data Analyst for AI Performance Evaluation - Remote

Overview

We're seeking a data-driven analyst to conduct comprehensive failure analysis on AI agent performance across finance-sector tasks. You'll identify patterns, root causes, and systemic issues in our evaluation framework by analyzing task performance across multiple dimensions (task types, file types, criteria, etc.).

In Short

• Conduct statistical failure analysis on AI agent performance.
• Identify root causes of failures in task design and evaluation criteria.
• Analyze performance variations across finance sub-domains.
• Create dashboards and reports for performance insights.
• Recommend improvements based on statistical findings.
• Communicate insights to stakeholders effectively.

Requirements

• Strong foundation in statistical analysis and hypothesis testing.
• Proficiency in Python or R for data analysis.
• Experience with exploratory data analysis.
• Understanding of AI/ML evaluation methods.
• Comfortable with data visualization tools and SQL.
• 2-4 years of relevant experience in data analysis.

Benefits

• Opportunity to work on cutting-edge AI projects.
• Collaborate with experts in the finance sector.
• Access to professional development resources.
• Flexible working environment.
• Competitive salary and benefits package.