Skip to main content
Search

# FPGA compute-NIC datapath

1 related articles

Subscribe to updates
  1. arXiv

    Long-context inference stops re-streaming KV cache: MCF replicates early blocks, cutting 16-node TTFT to 0.62 s

    Prefill in long-context inference repeatedly moves historical KV cache, and MCF turns KV layout into a stage-aware runtime schedule: early in prefill, when memory headroom exists, it temporarily replicates highly reused KV blocks, drains them as memory pressure rises, and converges to a sharded layout before decode. On a cluster of four KR260 FPGAs at 16K prefill with 16 logical nodes, MCF reaches 0.62 s TTFT, versus 1.91 s for Ring Attention and 1.81 s for Zigzag Attention.

You’re all caught up