MemT-1.0 folds memory into model computation: at 128K history its persistent state stores about 1/66 of a full-context KV cache
Synopsis
The MemTensor team released MemT-1.0, a memory foundation model that makes persistent, dynamically evolving memory an intrinsic part of model computation, so inference no longer reads the original context. It combines dynamic local memory with trainable hyper memory, and its persistent state stays constant in size, storing roughly 66 times less than a full-context KV cache at 128K history length. Across 11 memory benchmarks, MemT-1.0-Flash and MemT-1.0-Nano rank first and second among the evaluated parametric-memory methods, while textual-memory systems remain ahead overall, particularly on long histories that require detailed evidence retrieval.
Interpretation
MemT-1.0 lets a model retain and use historical information without re-reading the original context at inference, and its persistent memory state stays constant in size. The prior route kept history in the context, carried by a full-context KV cache that grows with history length. At 128K history length, the persistent state reduces storage by approximately 66 times versus a full-context KV cache.
Memory is carried by a native memory block that runs alongside backbone attention, and its local memory block maintains a multi-channel memory matrix that evolves across interaction steps. The hyper memory block supplies trainable memory query, key and value projections, erase, write and forget gates, and an output gate. During generation, memory attention reads the current memory state, normalizes and gates the readout, and adds it to the backbone attention output; at the end of each turn or segment, GDN-2 performs token-level memory updates.
The model family has two sizes, MemT-1.0-Nano and MemT-1.0-Flash, built on Qwen3.6-35B-A3B and DeepSeek-V4-Flash backbones respectively. Nano has 36B total parameters, 3B activated, 451M trainable memory parameters and a 10M dynamic memory state; Flash has 288B total, 17B activated, 3.61B trainable memory parameters and a 336M dynamic memory state. Training and evaluation use 34B tokens of data and dedicated infrastructure, covering 11 memory benchmarks.
Perspective
The demonstrated scope is a comparison among parametric-memory methods on 11 memory benchmarks and a storage comparison at 128K history length; the example deployment targets a Linux server with NVIDIA CUDA and Python 3.11 or newer, and the Nano example uses two GPUs.
On the 11 memory benchmarks, MemT-1.0-Flash and MemT-1.0-Nano rank first and second only among the evaluated parametric-memory methods, and textual-memory systems remain ahead overall, particularly on long histories that require detailed evidence retrieval; the roughly 66-fold storage reduction holds at the 128K history length condition.
