You didn’t type 3.77 billion tokens.

The tiniest part of the bill is the part you typed.

Every time you hit enter, the model doesn’t “remember” anything — the entire conversation gets wrapped up and sent again from the top.That’s how LLMs fake memory. So your first message costs what you typed. Your second costs what you typed plus the answer plusthe first message. By your 30th, the thing you actually just wrote is a rounding error, and everything before it is the bill.

That old material has a name: reused input— the part of every request the model has already seen. And when an answer fails and you say “no, fix that,” the retry carries all of it a second time, a third, a fourth.

The labs won’t fix this — the incentive runs the other way.

There’s a hope going around that bigger context windows and smarter models will make this clean itself up — that you can keep a messy desk and it’ll be fine. It won’t. Ironically, the morecapable the tool and the more tools you hand it, the more it piles onto the desk and the faster you hit the wall. Reducing your token bill is not something any lab is graded on. It’s your desk. You own it.

The habits that actually move the needle.

• Edit your mistake, don’t argue with it. A wrong answer usually means an unclear request — hit edit and fix the original, don’t add a “no, that’s not it” turn that re-sends everything.
• Start a clean task when the job changes. This drove the single biggest measured drop. Long threads are great while you’re on one problem; they’re brutal once the job shifts. Keep the old one — just stop making the next job carry it.
• Carry the artifact, not the argument. When a stage produces a result (a research report, a spec), carry that into the next step — not the bad first draft, the three rounds of critique, and the rejected sources.
• Ask for only what you need. Output is paid four times over — once to write it, then again as input on every turn after. “Give me 50 words” beats “write me 50 pages” on every future message too.
• Load only the tools the job can use. A typical multi-server setup (GitHub + Slack + Sentry + Grafana) burns ~55,000 tokens of tool definitions before the model does anything — Anthropic’s own figure.
• Keep answers where you can look them up. A place you (and the AI) can retrieve a past result beats re-deriving it from scratch every time.

How we build, so the desk stays clean.

We don’t just preach this — the agent system we run on is built around it, and it’s the reason it doesn’t drown in reused input:

• Fresh session per task, not one endless thread — the “start clean” rule, enforced.
• File hand-offs, not conversation history — each step reads a small compiled file (the artifact), never the whole prior thread. That’s compiled AI applied to context.
• A knowledge store it retrieves from — decisions and answers are saved once and looked up, not re-derived.
• Hard token limits and only-the-needed-tools loaded per step.

None of that is exotic. It’s desk hygiene turned into architecture — and it’s exactly the Cost Engineering Playbook a team can adopt.