Skip to main content
Search

# Memory Consolidation

1 related articles

Subscribe to updates
  1. arXiv

    Pull the prompt and the gain vanishes: Cortex writes context into feed-forward weights, keeping 88.9% of the gain after 50 sequential edits

    Large language models perform better when new knowledge sits in the prompt, but that gain usually disappears once the prompt is removed. Cortex distills the context-conditioned behavior into low-rank increments or added nonlinear channels inside feed-forward networks, using queries the model generates itself to anticipate how the context will be used. After 50 sequential factual edits on a 14B model, Cortex-Structural recovered 88.9% of the context-induced gain while retaining 99.8% of non-target performance.

You’re all caught up