Skip to main content
Search

Mi Zhang

1 related articles

Subscribe to updates
  1. arXiv

    FP4 reinforcement learning stops quantizing each path alone: TRACE shares rounding across training and rollout for up to 5.4x faster generation

    Reinforcement learning for mixture-of-experts language models can generate rollouts in FP4 to save compute and memory, but quantizing the training and rollout paths separately sends the same activation to different FP4 codewords, amplifying error into policy mismatch and unstable training. TRACE feeds the FP4 quantization outcomes actually produced during rollout back to the training side to choose how training activations are rounded, and caches only mantissa and scale information from deeper layers to keep transfer costs down. Across four Qwen mixture-of-experts models it supports FP4 weights, activations and KV cache together, matches BF16 rollout on average score, and speeds rollout by up to 5.4x.

You’re all caught up