September 2026 · Cost Watch
The Open Model Map.
AT&T now pushes tens of billions of tokens a day and holds costs flat by routing a growing share through open models. The headline everyone will repeat is “90% cheaper.”
The useful question is different: which of their moves survive at your size — and which quietly don’t. Here’s the map.
Built on others’ work
Enterprise figures come from AT&T’s own tokenomics program — via Fierce Network’s interview with Mark Austin, VP of Data Science at AT&T, their “Tokenomics Equation” post and 2026 trade-press interviews (Fierce Network) — the same team whose TM Forum MoDaaS research underpins our cache-trap breakdown. Use-case verdicts below come from our own research pass, dated 2026-09-01. We didn’t run AT&T’s experiments; they did, at a scale nobody reading this operates at. That gap is the point of this page.
What AT&T actually reported (as of September 2026)
How: a proprietary cache-aware router picking the cheapest adequate model per task, plus fine-tuning their own open telco models (OTel 2.0) where generic ones missed. AMD’s account of the same program is quotable verbatim: “Its cache-aware AI gateway processes an average of 45 billion tokens daily and has reduced AI costs by as much as 80%” (AMD newsroom, Jul 23, 2026). The two accounts of the reduction disagree — AMD’s newsroom says “as much as 80%”, Fierce’s reporting says up to 90% — so we show the spread rather than picking the flattering end. Note what the headline hides: the 90% is on the work that moved, and it took an in-house router and a training program to move it. T-Mobile’s public take on the same trend is the right caption for this whole page: open models are about fit, not just cost.“We’re trying to keep our overall cost in control.”
Three dials, not one decision
“Should we use open models?” conflates three independent choices. Teams that keep them separate make better calls than teams that argue about “open vs. closed.”
Dial 1 · Weights
Open ↔ Closed
Who can inspect, fine-tune, and re-host the model. Licensing and vendor-dependence live here.
Dial 2 · Hosting
API ↔ Self-hosted
Who runs the serving. Privacy posture, ops burden, and the serving footprint live here.
Dial 3 · Tier
Frontier ↔ Budget
How much capability the task actually needs. Most of the money is won or lost here.
The combination most people miss: open weights served by someone else’s API. That’s where most of AT&T-style savings actually live, and it requires no hardware, no MLOps, and no memory math. Two of the five tiers in our own calculator ladder are open-weight models served as APIs today: DeepSeek V3.2 and Qwen3.7 Flash — at $0.259/M input against $5/M for Claude Opus 4.8. Self-hosting is a separate decision with its own honest math — we wrote that one up too, and it’s less romantic than the demos.
Where open models are ready — verdicts as of 2026-09-01
From our September 2026 research pass across measured benchmarks and hands-on reports. These verdicts are dated on purpose: this table rots, and an undated version of it would eventually lie to you.
| Workload | Open-model verdict |
|---|---|
| Transcription | Closed — open options lead paid APIs |
| Routine drafting (email, letters) | Effectively closed at 70B class |
| Clean-document summarization | Largely closed |
| Embeddings / RAG over your documents | Good enough, and far cheaper at volume |
| Extraction from dense / scanned docs | Gap is real — and it fails quietly |
| Long-context reasoning (100k+) | Degrades from ~32k, under-appreciated |
| Agentic multi-step / hard reasoning | Narrowing, not closed at the top |
The translation: what survives at 1/10,000th of AT&T’s volume
Survives — do these
- • Tier substitution — route the 60–80% of work that never needed frontier to an open-weight API tier. This is most of the 90%, and it needs no infrastructure.
- • Prompt caching & batching — the API-side discounts stack to ~95% off list on batchable work; you don’t need AT&T’s router to claim them.
- • Route at the task level, not mid-conversation — their router is cache-aware for a reason: naive mid-session switching measured 4.9× MORE expensive in the same team’s earlier research.
Does not survive — skip these
- • Training your own model — OTel 2.0 exists because AT&T has telco-specific accuracy needs and 45B tokens/day to amortize it. At SMB volume, fine-tuning rarely beats a better prompt on a bigger open model.
- • Building a proprietary router — a routing rule in a config file gets you most of the win. The router is their scale talking, not a prerequisite.
- • Assuming “open = run it yourself” — self-hosting has its own economics (serving footprint, ops, utilization) that mostly favor the API form of open models until your volume is large and steady.
Where open still loses (we’d be selling something if we skipped this)
- • Tokens-to-answer: a cheaper price per token buys nothing if the model needs three attempts. Total cost = price × tokens-to-answer × serving footprint — our open-source cost breakdown shows a heavy open model losing on all three at once.
- • Quiet failure on messy inputs: extraction from degraded documents fails by inventing values, not by erroring. The workloads where errors are expensive are the workloads to keep on the strongest model you can justify.
- • Non-English work: hands-on 2026 reports document severe quality drops in smaller languages that benchmark tables don’t show.
- • The hard 20%: AT&T’s own target keeps 30–35% of tokens on closed frontier models. The company saving 90% on routed work still pays list price for judgment. So should you.
The Scruuge take
The biggest AI buyer news of 2026 isn’t a model launch — it’s a Fortune-50 CFO story: volume up 5×, cost flat. But the lesson is not “open models are cheaper.” It’s that routing is the discipline and open models are the destination for the easy majority of your tokens — while the hard minority stays expensive on purpose. Turn three dials separately, date your verdicts, and re-check them: this map was drawn in September 2026, and the territory moves.