Researcher / Engineer in Generative Models Optimization - Remote

Overview

Black Forest Labs is a cutting-edge startup pioneering generative image and video models. Our team, which invented Stable Diffusion, Stable Video Diffusion, and FLUX.1, is currently seeking a strong researcher / engineer to work closely with our research team on pretraining and inference optimization.

In Short

• Finding ideal training strategies for various model sizes and compute loads.
• Profiling, debugging, and optimizing single and multi-GPU operations.
• Reasoning about speed and quality trade-offs of quantization.
• Developing low-level kernel optimizations for inference and training.
• Innovating ideas to maximize GPU capabilities.

Requirements

• Familiarity with techniques in optimizing inference and training workloads.
• Experience optimizing memory-bound and compute-bound operations.
• Understanding GPU memory hierarchy and computation capabilities.
• Deep understanding of efficient attention algorithms.
• Implementing Triton kernels and ensuring correctness.
• Using pybind to integrate custom kernels into PyTorch.

Benefits

• Experience with Diffusion and Autoregressive models is a plus.
• Experience in low-level CUDA kernel optimizations is a plus.