Staff Software Engineer, Speculative Decoding - Remote

Overview

Groq delivers fast, efficient AI inference. Our LPU-based system powers GroqCloud™, giving businesses and developers the speed and scale they need. Headquartered in Silicon Valley, we are on a mission to make high performance AI compute more accessible and affordable. When real-time AI is within reach, anything is possible. Build fast.

In Short

• Design, implement, and optimize speculative decoding algorithms.
• Collaborate with cross-functional teams for integration into production AI infrastructure.
• Work in a multi data center production environment with Kubernetes.
• Develop high-performance, scalable code in C++ and Rust.
• Experience in building production distributed systems.
• Stay updated with developments in generative AI.
• Drive improvements in model evaluation and system performance.
• Provide technical leadership and mentorship.
• Champion code quality and best practices.

Requirements

• Master’s degree in Computer Science or related field.
• Extensive experience in generative AI inference.
• Proficiency in C++.
• Strong analytical and problem-solving skills.
• Ability to work in fast-paced environments.
• Understanding of Generative AI model architecture.
• Familiarity with AI infrastructure challenges.

Benefits

• Competitive base salary.
• Comprehensive compensation package including equity.
• Diverse and inclusive workplace.
• Opportunities for professional growth.
• Collaborative work environment.