How to save tokens when working with AI agents: 7 tactics with numbers

Short version: tokens are eaten not by the agent’s work but by repeatedly feeding it context. Seven tactics that work: memory instead of re-explaining the project, targeted search instead of whole files, a short system prompt, session discipline, matching the model to the task, prompt caching, and measuring usage. On our own repo this cut session-start tokens by 1.4–3.3×, and up to 4.2× on pinpoint questions.

Where the tokens actually go

Before saving, look at the anatomy of the bill. Every agent request pays for four buckets:

1. The system prompt — rules, instructions, tool descriptions. Paid on every single request, no exceptions.
2. Session history — the entire dialogue so far, including the stale parts. It grows in spirit quadratically: the longer the session, the more expensive each next turn.
3. Files and code — the heaviest bucket for coding agents: “read the repository” easily means tens of thousands of tokens.
4. Tool outputs — logs, search results, diffs. Often returned in full when two lines would do.

All seven tactics below hit these buckets, in descending order of impact.

1. Memory instead of re-explaining the project

The most expensive habit is re-telling the project every session. The stack, the conventions, “yesterday we fixed that bug” — that’s thousands of tokens a day, typed by hand.

We measured it on our own repository: starting a session with a memory digest costs ~4,200 tokens; reading the project from scratch costs ~8,900. A 2.1× difference on every start (methodology and full numbers). Memory here means entities, decisions and open tasks the agent reads in one budget-capped call, instead of rescanning files.

That’s what MaxGraph is for: a local MCP server that keeps a memory graph and serves a digest at session start. It works in the trial and installs with one prompt.

2. Targeted search instead of whole files

“What’s the status of the migration task?” doesn’t require reading the changelog. A question spread across two files costs ~4,300 tokens of reading versus ~1,000 for one memory search — up to a 4.2× difference.

The practical rule: if the answer is a fact or a decision, the agent should search memory or code (grep, semantic search), not read files whole. Reading a file in full is justified only when you’re editing that file.

3. A short system prompt

Every line in a system prompt is paid on every request. From practice: periodically re-read your CLAUDE.md / AGENTS.md and cut the rules that never once fired. A “just in case” rule costs you daily; a “needed it once” rule doesn’t.

A good test: if the agent violated the rule at least once this month, keep it. If it never fired and never helped, it’s a deletion candidate.

4. Session discipline: close, don’t drag

A long session is history you carry into every request. By the end of the day, a typical coding-agent session is so heavy that each new question costs as much as a short article.

Better: finish the phase, record the outcome (our session_close writes it to memory), and start the next session fresh plus the digest. Context compaction is the same mechanism in emergency mode: better to close the session before the client starts truncating it for you. More on that: what context compaction is.

5. Match the model to the task

Routing “read this log and tell me if it crashed” doesn’t need a flagship model. If your stack allows per-task model choice, hand routine reads and classifications to the cheap model and keep the expensive one for design and debugging. An order-of-magnitude saving per token, with no quality loss where it matters.

6. Prompt caching

If your provider supports prompt caching (Anthropic, OpenAI and others cache the repeated request prefix), a stable system prompt and a stable context order earn a real discount on the cached portion. One condition: the request prefix must be byte-identical between calls. A fixed-structure memory digest (like ours) caches better than a free-form re-telling.

7. Measure, or it’s guesswork

You can’t save what you don’t measure. Minimum: count tokens per session start and per typical question before and after a change. We keep these measurements on ourselves and publish them as standalone articles, because “it got cheaper” without a number is not an answer.

FAQ

Does saving tokens mean losing quality? No, if you cut repetition rather than signal. Memory returns exactly what’s relevant: decisions, open bugs, tasks. Reading “everything” adds noise, not quality — and on long contexts the model finds the middle less reliably anyway.

How much can I realistically save? On our repo: 1.4–3.3× fewer tokens at session start, up to 4.2× on pinpoint questions. That’s a measurement, not a promise: the methodology is open — rerun it on your own project.

When should I NOT save? When the context is the work: reviewing a big diff, analyzing a long document. Whole-file reading is justified there, and there’s no cheaper way. Saving is about recurring costs, not one-offs.

What’s next

• The numbers behind this article: we measured the cost of re-explaining context — tables and caveats included.
• Start with memory in 10 minutes: memory for Claude Code or for Cursor.
• Terminology: the agent memory glossary.