Skip to main content
Search

# vLLM

2 related articles

Subscribe to updates
  1. vLLM Blog

    vLLM folds decode layers into a few kernel launches, lifting single-user interactivity by up to 2.26x

    AMD has brought FlyDSL-written MonoKernels into vLLM, merging the many operator launches of a decode layer into a few persistent kernel launches that reuse thread blocks across stages, prefetch weights and pass ready data through tagged mailboxes. On MI355X, DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, MiniMax-M3, Kimi-K3 and Muse Spark improve concurrency-1 interactivity or time per output token by 1.20x to 2.26x, with DeepSeek-V4.1-Flash also merged on MI325X, and the largest gains at low concurrency. The path is off by default, enabled with --monokernel, and serves only specific models, tensor-parallel sizes and step widths.

  2. vLLM Blog

    vLLM splits multimodal encoding into its own instances: 6.6–9.5% more throughput per GPU on DeepSeek V4.1 Flash

    vLLM now lets multimodal serving deploy encoding, prefill and decode as separately scaled instances (EPD), so encoder capacity for images, video and audio can grow independently of the language model. The release adds metadata-only request forwarding, a dedicated encoder execution path, remote embedding transfer and cross-encoder cache reuse. On DeepSeek V4.1 Flash document question answering at concurrency 64–128, input-plus-output throughput per GPU across all 12 EPD GPUs was 6.6–9.5% above an eight-GPU PD baseline and P90 time-to-first-token was 44–47% lower, while at concurrency 16 EPD was slower.

You’re all caught up