Skip to main content
Search

# Humming

1 related articles

Subscribe to updates
  1. LMSYS Org

    Humming 0.1.18 lands in SGLang: at matched A16 precision, long-context prefill runs 1.07–1.23× faster than Marlin

    Humming 0.1.18 connects low-bit quantized kernels to SGLang's MoE inference path, covering more quantization schemes and both Hopper and Blackwell GPUs, and adds a shared activation-expression interface. On DeepSeek-V4.1-Flash-1017 long-context tests, prefill at matched A16 precision is 1.07–1.23× faster than Marlin; at EP1 and A16, decode throughput ranges from 2.89% lower than Marlin at concurrency 1 to 7.65% higher at concurrency 64. These end-to-end results cover only that checkpoint and the H20-3e environment.

You’re all caught up