Skip to main content
Back to timeline
arXivSource publication:

Long-horizon exploration memory gets a graph benchmark: LExMem spans 224 dialogues, and LExGraph tops seven of ten memory abilities

Synopsis

Long-term memory evaluation has mostly tested factual recall and monotonic updates, leaving out the dependencies among branches, revisions and revivals in an exploration process. LExMem models exploration as directed graphs, raises their structural complexity through an evolutionary mechanism, and instantiates them as research and software-design tasks, yielding 224 dialogues and 2,018 question-answer instances. The companion LExGraph incrementally builds a memory graph from raw dialogues and gathers evidence through hybrid retrieval plus neighbor expansion; it posts the highest overall result on LExMem and leads seven of ten memory abilities on the 128K subset of BEAM with an overall score of 0.472.

Source-provided article image: Twists and Turns: A Graph-Based Benchmark and Framework for Long-Term Memory in Exploration Tasks
arXiv

Interpretation

Long-term memory evaluation moves from remembering facts to remembering an exploration path: questions no longer only ask what a piece of information is, but require stitching scattered evidence along a chain of attempts that were revised, refuted and revived. Earlier long-term memory benchmarks centered on recalling user personas and personal facts, or on tracking cross-session task instructions and project states, and still measured information recall. LExMem comprises 224 dialogues and 2,018 QA instances, with dialogues ranging from 51K to 174K tokens and averaging 88K; questions are derived from three graph structures, dense subgraphs, sparse subgraphs and same-type node sets, contributing 561, 759 and 698 instances respectively.

The memory framework organizes information by the progression of the dialogue rather than by semantic similarity or entity relations alone. Existing memory frameworks organize dialogue history around semantic similarity, topical clustering and entity relations, without explicitly preserving the ordering dependencies among exploration steps. On LExMem, LExGraph achieves the highest overall performance under both the Qwen-3.8 27B and GPT-6 Luna backbones, ahead of baselines such as GraphRAG and LightMem; on the 128K-token subset of BEAM it ranks first in seven of ten memory abilities with an overall score of 0.472.

The memory graph is built incrementally from raw dialogues, with each node a key holding a summary plus salient facts and a value holding the raw turns, and with two edge types, hierarchical parent-child and cross-branch associative. Hierarchical edges follow the right-frontier constraint from discourse theory, so a new node can attach only along the path from the root to the most recent utterance, while cross-branch edges are selected from historical nodes outside that path by key-embedding similarity. Answering first fuses semantic cosine similarity and BM25 lexical scores by reciprocal rank fusion, takes a primary node set and then expands to its neighbors, and assembles context from primary keys and values, neighbor keys and the relation descriptions of connecting edges.

Perspective

The evaluation targets question-answering memory over synthetic dialogues in two domains, research exploration and software design, with dialogues between 51K and 174K tokens; code and data for the benchmark and framework are released.

The dialogues are synthesized from instantiated exploration graphs and may not fully capture the stylistic diversity of real conversations; the evaluation measures memory question answering about the exploration process, not whether stronger memory yields better exploration decisions, which is left to future work.