Data Scientist – AI & Agentic Applications & Benchmarking - Remote

Overview

The Data Scientist role at CloudBees focuses on driving insights and storytelling behind metrics related to AI and Agentic Applications.

In Short

• Work closely with engineers and product teams to prototype and measure AI experiences.
• Define, measure, and evangelize the impact of Agentic Applications.
• Use Python notebooks to explore data and visualize insights.
• Write internal documentation and performance summaries.
• Partner with teams to develop telemetry and eval frameworks.
• Help define KPIs and success metrics for AI features.
• Stay updated with metrics and evaluation techniques in AI.
• Lead as a builder, evaluator, and communicator in the team.
• Experience in startup or high-growth environments preferred.
• Comfort in fast-paced, ambiguous environments.

Requirements

• 3+ years of experience in data science or ML analytics roles.
• Proficiency in Python and experience with Jupyter notebooks.
• Experience with evals, telemetry, and evaluating ML systems.
• Ability to partner with technical teams for meaningful metrics.
• Clear communication skills for writing about outcomes.
• Experience with AI/ML tools like MLFlow and Hugging Face.
• Experience with agentic or LLM-based applications preferred.
• Familiarity with tools like LangSmith or OpenInference preferred.
• Background in developer tools or DevOps preferred.
• Comfort in fast-paced environments.

Benefits

• Base Pay Range: $166,000 – $180,000 annually.
• Health, Dental, and Vision Insurance.
• Short & Long Term Disability Insurance.
• Remote Work Environment.
• Flexible Time Off and Paid Company Holidays.
• Parental Leave and Variable Bonus Plan.
• Stock grant opportunities.
• 401(k) with Company Match.