Skip to main content
Search

Muralidhar Andoorveedu

1 related articles

Subscribe to updates
  1. arXiv

    A pipeline inside one GPU: SubGPU cuts a 36-layer decode iteration by 10.33% and lifts DRAM throughput to 83.6% of peak

    LLM decode is dominated by many short, memory-bound operators, and a single operator often hits the HBM bandwidth ceiling before it uses all streaming multiprocessors (SMs), while tensor, data, expert, context and pipeline parallelism only divide work across GPUs and leave this intra-GPU inefficiency untouched. SubGPU Pipelining cuts the operator sequence of a layer or repeating graph into ordered contiguous segments, binds each segment to a disjoint SM pool, and streams microbatches through the stages. On GPT-OSS 120B under 32-GPU B200 deployment settings, a hardware experiment using one B200 to emulate the per-rank workload reduced the 36-layer decode iteration from 14.970 ms to 13.424 ms and raised mean DRAM read-plus-write throughput from 71.8% to 83.6% of peak.

You’re all caught up