When we create a large amount of memories or research docs I always find it best to create a table of contents or a reference doc citing all the docs and a short sentence about it.
Then that TOC becomes the "required read" before everything in any new agent. Which increases the memory doc read substantially since it understands what each doc is natively.
The agent in this run did exactly that, natively. It maintained MEMORY.md as a table of contents: one line per note with a name and a short description, and that index was spliced into the system prompt of every session in every memory arm, so each new session started already knowing what every note was about. The note bodies sat one read away.
What happened: sessions opened a note body in 101 of the 1,080 held-out runs that had them, and mounting the index and files moved nothing across nine comparisons. So the required-read TOC pattern was implemented here by the agent itself, and it did not convert into improvement.
That may say more about this setting than about your pattern: every attempt was graded and the whole record was still in context, so the notes had nothing to add that the conversation did not already carry.
You are right, context helping is known. The investigation started with what the agent reached for when asked to get better. It could have reached for code execution, subagents, web search, or memory. It reached only for memory and spent the run building a knowledge base, so the natural reading was that the notes were the mechanism. The ablations say otherwise. Fresh sessions were booted from each element of the context and validated on the same 120 held-out tasks: the notes moved nothing in nine comparisons, the thought content decayed to nothing across the run, and the record of graded tasks with post-compaction summaries carried the improvement. As an artifact, the memories Claude Code wrote for itself had no effect on the improvement it was making. I think that is interesting.
When we create a large amount of memories or research docs I always find it best to create a table of contents or a reference doc citing all the docs and a short sentence about it.
Then that TOC becomes the "required read" before everything in any new agent. Which increases the memory doc read substantially since it understands what each doc is natively.
Post author here. Thanks for reading!
The agent in this run did exactly that, natively. It maintained MEMORY.md as a table of contents: one line per note with a name and a short description, and that index was spliced into the system prompt of every session in every memory arm, so each new session started already knowing what every note was about. The note bodies sat one read away.
What happened: sessions opened a note body in 101 of the 1,080 held-out runs that had them, and mounting the index and files moved nothing across nine comparisons. So the required-read TOC pattern was implemented here by the agent itself, and it did not convert into improvement.
That may say more about this setting than about your pattern: every attempt was graded and the whole record was still in context, so the notes had nothing to add that the conversation did not already carry.
The index the agent built is here, with the note bodies beside it: https://github.com/shojin-lab/shorep/blob/main/studies/conve...
I don't get the point: Claude gets better with session context? Doesn't everyone know that already?
Thanks for reading! I am the author of this post.
You are right, context helping is known. The investigation started with what the agent reached for when asked to get better. It could have reached for code execution, subagents, web search, or memory. It reached only for memory and spent the run building a knowledge base, so the natural reading was that the notes were the mechanism. The ablations say otherwise. Fresh sessions were booted from each element of the context and validated on the same 120 held-out tasks: the notes moved nothing in nine comparisons, the thought content decayed to nothing across the run, and the record of graded tasks with post-compaction summaries carried the improvement. As an artifact, the memories Claude Code wrote for itself had no effect on the improvement it was making. I think that is interesting.