Senior Software Engineer - Distributed Inference - Remote

Overview

NVIDIA is seeking a Senior System Software Engineer to develop user-facing tools for the Dynamo Inference Server, focusing on large-scale AI inference workloads.

In Short

• Build and maintain distributed model management systems.
• Implement inference scheduling and deployment solutions.
• Collaborate with engineers and researchers on scalable APIs and services.
• Create monitoring, benchmarking, and automation processes.
• 6+ years of professional systems software development experience required.
• Strong programming expertise in Rust, with C++ and Python as a plus.
• Deep knowledge of distributed systems and runtime orchestration.
• Hands-on experience with Kubernetes and container-based microservices.
• Proven ability to excel in fast-paced R&D environments.
• Experience with inference-serving frameworks is a plus.

Requirements

• Bachelor’s, Master’s, or PhD in Computer Science, ECE, or related field.
• Experience deploying/managing LLM inference pipelines at scale.
• Strong background in GPU inference performance tuning.
• Contributions to large-scale, low-latency distributed systems preferred.
• Ability to work across cloud-native and hybrid environments.

Benefits

• Competitive salary based on experience and location.
• Eligible for equity and benefits.
• Diverse and supportive work environment.
• Opportunity to make a lasting impact in the AI field.
• Work with some of the most forward-thinking and committed people.