Inference Optimization Engineer - Remote

Overview

As an Inference Optimization Engineer, you will improve the speed and efficiency of large language models at the GPU kernel level, through the inference engine, and across distributed architectures.

In Short

• Identify bottlenecks and optimize inference efficiency.
• Build repeatable tests that model production traffic.
• Reduce memory use and compute cost with mixed precision.
• Improve batching, caching, load balancing, and model-parallel execution.
• Write technical posts and contribute to the open-source community.

Requirements

• Deep understanding of transformer architecture.
• Hands-on experience with model serving optimizations.
• Experience with inference engines like vLLM, SGLang, or TRT-LLM.
• Proficiency in CUDA and profiling tools.
• Track record of blog posts or conference talks in ML systems.

Benefits

• Direct impact on distributed LLM inference.
• Work remotely from anywhere.
• Competitive salary and equity.
• Learning budget and paid conference travel.