Nicolas Gras (@armgd.bsky.social)

δ-mem: add a tiny 8×8 online associative memory to a frozen full-attention transformer. Update it with a delta-rule, read it out to generate low-rank attention corrections. Reports ~1.10× avg gain; bigger on memory-heavy MemoryAgentBench/LoCoMo. Self-contained “memory” without longer context or fine

loading . . .

$δ$-mem: Efficient Online Memory for Large Language Models Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply expanding the context window is costly and often fails to ensure effective context utilization. We propose $δ$-mem, a lightweight memory mechanism that augments a frozen full-attention backbone with a compact online state of associative memory. $δ$-mem compresses past information into a fixed-size state matrix updated by delta-rule learning, and uses its readout to generate low-rank corrections to the backbone's attention computation during generation. With only an $8\times8$ online memory state, $δ$-mem improves the average score to $1.10\times$ that of the frozen backbone and $1.15\times$ that of the strongest non-$δ$-mem memory baseline. It achieves larger gains on memory-heavy benchmarks, reaching $1.31\times$ on MemoryAgentBench and $1.20\times$ on LoCoMo, while largely preserving general capabilities. These results show that effective memory https://arxiv.org/abs/2605.12357