Skip to main content
Search

Ruijia Yang

1 related articles

Subscribe to updates
  1. arXiv

    SlideDP speeds shared-host multi-GPU full-parameter fine-tuning by 1.46–2.64x and beats FSDP2's measured peak by 11.2% at a larger batch on four H100s

    SlideDP introduces a synchronous data-parallel runtime for shared-host multi-GPU systems that maintains one authoritative host state, decouples communication routes from state layout, and pipelines parameter delivery, gradient aggregation, and CPU updates across ranks and chunks, achieving geometric-mean throughput ratios of 1.46–2.64 over SlideFormer, MegaTrain, and ZeRO-Offload in matched-batch sweeps, approaching GPU-resident FSDP2 at a smaller batch and exceeding its measured peak throughput by 11.2% at a larger batch on four H100s, while supporting 256K-token sequences for Qwen3-14B and fine-tuning Qwen2.5-72B on four RTX 4090 GPUs.

You’re all caught up