Thinking Machines releases looped model Inkling 2 Mini: its middle 12 layers run three times, and vLLM supports it on day 0
Synopsis
Thinking Machines released Inkling 2 Mini, a multimodal mixture-of-experts model whose layers 6 through 17 run three passes with shared weights, each pass keeping its own KV cache and reading gated messages from the pass before. vLLM supports it on day 0 on NVIDIA Blackwell, Hopper and DGX Spark, managing each pass's caches like any other cache layer and reusing Inkling 2 Small's fused kernels. The same release includes TML-d24-varloop-research, a research checkpoint whose number of passes and KV-cache sharing are chosen at startup.
Interpretation
Inkling 2 Mini splits its 24 layers into a prelude, a loop core and a coda; the core's layers 6 through 17 run three passes with shared weights, and the coda continues from the last pass. A plain looped model feeds pass p's output into pass p+1, whereas here all three passes start from the same prelude output and a later pass sees the earlier one only through messages. From the second pass on, each core sublayer's attention query, key and value inputs and its MLP input each read a message, so with 12 core layers and 4 inputs each, every token reads 48 messages in later passes.
vLLM registers each extra pass of a core layer as its own attention layer with its own KV cache and conv-state cache, so a 24-layer model carries 48 KV caches and 48 conv-state caches. The KV cache manager and prefix caching treat these caches like any others, so serving the loop needed no new cache-management logic. One attention-residual kernel reads all four messages of a layer at once, the four controllers' weights are stacked into batched GEMMs, and the merge runs in FP32 without FMA contraction, matching the reference.
The same release includes the research checkpoint TML-d24-varloop-research, whose number of loop passes and whether passes share one KV cache are selected at startup through --hf-overrides. The fixed-loop Inkling 2 Mini rejects both options; with a shared cache each pass overwrites the keys and values of its tokens so later tokens read the last pass's, while conv states stay per pass. On one GB200 at TP1, 4K context and forward pass only, a decode step at batch 1 takes 1.51, 2.67 and 3.85 ms for 1, 2 and 3 passes, with KV cache capacity of 352K, 233K and 175K tokens; three passes with a shared cache give 3.86 ms and 194K tokens.
Perspective
The demonstrated scope is vLLM's serving support for Inkling 2 Mini and TML-d24-varloop-research on NVIDIA Blackwell, Hopper and DGX Spark, with the image vllm/vllm-openai:inkling-2-mini; the performance figures come from one GB200 at TP1, 4K context and forward pass only.
The variable-loop checkpoint's trade-off is measured only on one GB200 at TP1, 4K context and forward pass only; sharing the KV cache frees less memory than dropping a pass because the per-pass conv-state caches remain. The DSpark draft requires the default three passes because it reads fixed layer visits.
