Inference accelerators can learn while serving: LWS hides learning in prediction's idle compute and keeps 99.3% of the quality benefit
When a long-context model keeps learning during inference, it pays for backward computation and weight updates on top of prediction, while today's inference accelerators assume fixed weights. LWS schedules that learning onto matrix capacity left idle during prediction, uses per-layer weight versioning so prediction can read slightly older weights, and lets long learning kernels pause and resume across fragmented idle windows. On modeled datacenter and edge NPUs, TTT-E2E 3B at 128K tokens sustains prediction-only throughput while preserving 99.3% of fully fresh learning's quality benefit.