Skip to main content
Search

Tingjie Chen

1 related articles

Subscribe to updates
  1. Phones and cloud split generative inference: MAGS bids instead of stampeding, cutting p99 latency 17.1%

    A generative query's cost is unknown until the answer is written, so devices reserve resources blind and many crowd one server at once. MAGS has each device forecast token count and generation entropy with a four-layer probe on its local model, then bid for cloud capacity against a broadcast congestion price, choosing its per-token privacy parameter in the same bid. Over ten seeds it cuts p99 latency 17.1%, energy per token 13.6% and raises throughput 20.7% against a reactive PPO baseline given identical observations; an eight-device, two-server testbed drops the cross-device correlation of offload decisions from 0.41 to 0.08.

You’re all caught up