Senior AI Systems Engineer (LLM Inference & Infra Optimization) - Remote

Overview

At Sully.ai, we’re building cutting-edge AI-native infrastructure to power real-time, intelligent healthcare applications. Our team operates at the intersection of high-performance computing, ML systems, and cloud infrastructure — optimizing inference pipelines to support next-generation multimodal AI agents. We're looking for a deeply technical engineer who thrives at the systems level and loves building performant, scalable infrastructure.

In Short

• Lead efforts in deploying and optimizing large language models on high-end GPU hardware.
• Work across the stack from C++ and CUDA kernels to Python APIs.
• Shape DevOps practices for scalable, multi-cloud deployments.
• Develop and optimize inference pipelines using quantization and attention caching.
• Build and maintain low-level modules in C++/CUDA/NCCL.
• Manage multi-cloud environments using IaC frameworks like Pulumi or Terraform.
• Design low-latency streaming and decision-support systems.
• Build robust tooling and interfaces for other engineers.

Requirements

• Proficiency in C++, CUDA, and Python.
• Deep understanding of GPU architectures and large model serving techniques.
• Hands-on experience with multi-cloud environments (GCP, AWS).
• Familiarity with ML deployment frameworks (TensorRT, vLLM).
• Comfortable with DevOps workflows and distributed system debugging.
• (Bonus) Experience with streaming embeddings or hybrid retrieval architectures.
• (Bonus) Interest in building tools for broader engineering teams.

Benefits

• Collaborate with a highly technical team solving hard problems in AI and healthcare.
• Work with cutting-edge GPU infrastructure.
• Be a foundational part of shaping AI-native infrastructure.
• Help accelerate a meaningful product that improves clinician workflows.