Is Your Agent's Memory Worth Its Tokens?

Agent memory has become the obvious upgrade. You give the agent a place to store what it learned, it recalls a relevant workflow next time, and the success rate on your benchmark goes up. The benchmark number goes up, and the upgrade looks obvious.

Memory is not free. Every workflow the agent stores and re-injects into its prompt costs tokens on the next step, and those tokens are no longer available for fresh reasoning. So the fair comparison is not "agent" versus "agent plus memory". It is "agent" versus "agent plus memory, both working under the same token budget". Most benchmarks never impose that budget, which means the memory agent quietly gets to spend more to reach its score.

I wanted to see what happens when you put the budget back. This article walks through a small, reproducible experiment on a local model, section by section, and ends with a short checklist you can apply to any memory claim you read. The runnable code lives in the GitHub repository.

Before measuring it, we should agree on what we are measuring. "Memory" is an overloaded word, so let us separate the common meanings. You might wonder which one this experiment targets, so let me name it. This experiment targets procedural memory specifically.

The mechanism we study is deliberately simple, because the point is the measurement, not the cleverness of the memory. After the agent solves a task, it saves the workflow it used. When a similar task arrives, that workflow is injected into the prompt as a hint. This mirrors the "store a solved workflow, replay it later" pattern behind popular memory modules, and the whole thing is just a few lines.

# After a task succeeds, store the workflow that solved it, keyed by task type.
if use_memory and ok and recipe:
    memory[task["type"]] = f"For '{task['type']}' tasks: {recipe}"

# On the next task of the same type, inject that workflow into the system prompt.
recipe = memory.get(task["type"])
if recipe:
    sys += f"\nA workflow that solved a similar task before:\n{recipe}"

To make the difference tangible, here is the same task through both agents' eyes. The question is "what is the cheapest audio item?", and both agents get the same tools and the same instructions. The only difference is the block of text at the bottom of the memory agent's prompt.

You are a tool-using shopping agent.
Tools:
  search(query) -> list matching items
  open(id)      -> full details of an item
  finish(answer)-> final answer, stop

Task: what is the cheapest audio item?
Begin.

You are a tool-using shopping agent.
Tools:
  search(query) -> list matching items
  open(id)      -> full details of an item
  finish(answer)-> final answer, stop

A workflow that solved a similar task before:
For 'cheapest_cat' tasks: search(<name>)
  -> open(<id>) -> open(<id>)
  -> finish(<the cheapest_cat>)

Task: what is the cheapest audio item?
Begin.

Injecting a stored workflow adds text to every step where it applies, and that text has a price paid in prompt tokens. On a task where the agent needs room to think through several tool calls, spending part of the budget on a recalled recipe leaves less room for the reasoning itself. The clean success-rate chart never accounts for that trade.

In the benchmark, that control is a single running counter. Every model call adds its tokens to the total, the recalled workflow included, and once the total crosses the cap the task fails. Both agents run through the exact same gate.

used = 0
for _ in range(max_steps):
    if used >= budget:              # the same cap applies to both agents
        return False, used, None    # out of budget, the task is a failure
    content, toks = call(msgs, temperature, seed)
    used += toks                    # every token counts, injected memory included

Without that control, a reported gain has two possible sources that nobody has separated: the memory genuinely helped, or the memory agent simply got to spend more tokens. A budget-matched comparison forces those two apart. It is one of the most useful things you can add to a memory evaluation, and it is usually missing.

To keep this reproducible on a laptop, I kept the setup small and self-contained. I ran everything on a local model through Ollama, and the agent code uses only the Python standard library, so there is nothing to install for the core benchmark and nothing hidden behind an API.

A result you cannot reproduce is an opinion, so here is the full protocol. Every number in this article comes from this exact configuration, and the one thing that changes between the two agents is whether memory is switched on.

The whole experiment is condensed into one script, and the notebooks in the repository unpack it one idea at a time if you want to follow along and change things.

Look at the means alone and memory wins: 56% against 54%. It is the kind of two-point bar-chart difference that ends up in a launch post. Now add the spread across the five seeds, using the toggle below, and watch what happens to that advantage.

When I ran it, the vanilla agent ranged from 30% to 80% across seeds; the memory agent from 40% to 70%. That two-point gap in the means sits inside a swing of roughly 25 points either side of the mean. On top of that, the memory agent spent about 2% more tokens to get there, because the recalled workflows cost prompt space.

So is memory useless? That would be the wrong lesson. A narrower reading survives: storing more text, and hoping recall lifts a score, does not hold up under a budget-matched test. What survives is a different kind of memory, and the recent literature points the same way.

In July 2026, six new agent-memory systems were published in a single week. Not one of them reported a budget-matched comparison against a plain agent, which is exactly the gap this experiment is about. But four of the six converge on the same idea:

All four point to the same idea: useful memory is structured, verified, and selectively exposed, not more raw history poured into the context. A verified procedure the agent can replay deterministically earns its tokens, while an undifferentiated pile of past text, recalled on similarity, mostly does not.

The takeaway is a measurement habit, one you can apply to any claim you read. Next time you meet "our memory module lifts success rate by X%", run the claim through four questions before you believe it.

None of these needs a big rig. The experiment behind this article fits in one standard-library file and runs on a laptop. The discipline is what transfers: hold the budget fixed, run several seeds, and compare the effect against the spread before you trust it.

This teaching benchmark does not prove that memory fails in general. It uses one small model, one synthetic tool-world, and one style of procedural memory. The absolute numbers would move with a larger model, richer tasks, or a smarter memory design.

What it does show, and what a production-scale study reported independently, is the direction and the method: under a matched budget, a plain agent is a much stronger baseline than the launch charts suggest, and variance can be much larger than the reported effect. Memory may help. You cannot know whether it does until you measure it against the right control.

Everything in this article is reproducible from the GitHub repository, which holds the agent, the tool-world, the benchmark, and the progressive notebooks.