The Context Tax: What AI Agent Context Really Costs

The Context Tax, Defined

A startup called Uvi raised $7.5 million to do something that sounds like a plumbing job: connect Slack, Jira, and Salesforce into one pool of company knowledge. The pitch deck would put you to sleep. The numbers wouldn't. Uvi claims its approach cut per-response costs from $0.75–$1.41 down to $0.10–$0.16, and cut response times in half.

Same models, roughly the same prompts. A seven- to nine-fold difference in what a single answer costs.

That gap is the context tax, the money, minutes, and mistakes you pay because the system doesn't know what your team already decided. It isn't a model problem. It's a filing problem wearing a model's clothes.

You've paid it. Everyone has. It's the fourth time this month you've pasted the same three paragraphs into a chat window to explain your architecture. (You could have written it once, in a file, and never again. You didn't.) It's the agent that confidently recommends a caching strategy your team rejected in March, so you burn twenty minutes re-litigating a decision that was already settled. It's the code review that catches nothing because the reviewer, human or otherwise, had no idea what the change was supposed to accomplish.

The tax lands in three buckets:

• Retrieval cost. Tokens and latency spent hunting for facts that should already be at hand.
• Correction cost. Human hours spent fixing output built on a stale premise.
• Re-explanation cost. The briefing you type again, and again, and again.

The third one is the expensive one, because nobody logs it. It doesn't appear on an invoice or a timesheet. It just quietly eats Thursday afternoon.

And it scales in a direction that surprises people. A solo developer pays it in minutes. A twenty-person team pays it in duplicated work and contradictory answers. An AI agent hooked into a messy knowledge base pays it in tool calls that go nowhere and permissions that block the right answer. The bigger your company gets, the more the tax compounds.

When was the last time you explained your own product to a machine for the eighth time this quarter?

The Receipt: Where the Money Actually Goes

The context tax is mostly spent on small, invisible failures: a tool call that returns the wrong ticket, a retry, a human re-prompt, a stale document. Individually, each costs pennies. At team scale, it's often the single largest line item in an AI budget.

Here's what it looks like in practice. A backend engineer gets paged at 11pm about a failed payment webhook. She asks her coding assistant to look at the handler. The assistant proposes a retry-with-backoff strategy, sensible, well-written, and exactly what the team rejected in March after the double-charge incident. The engineer now has to spend twenty minutes reconstructing why, because the reasoning lives in one Slack thread from a quarter ago and two people who've since left.

The model was fine. The memory wasn't.

Break the bill down and it looks like this:

1. Duplicate search. Three tool calls to find a ticket that a colleague could have named in four seconds.
2. Stale premises. Output built on documentation last touched in 2024.
3. Permission friction. The agent can't see the answer, so it guesses.
4. Human re-prompting. You, typing "no, we use X for this, not Y" for the third time.
5. Silent wrongness. The answer that sounds right, gets shipped, and gets fixed two sprints later at full price.

Number five is the killer. Everything else costs time. That one costs trust.

There's a second, quieter cost too: the attention of the person doing the correcting. Uvi's pitch is essentially that connecting systems halved response times. But the number worth staring at isn't the latency. It's that somebody used to spend their afternoon being the human connection between Slack, Jira, and Salesforce. Cheap answers don't help if a person has to assemble the context first.

Cheap Models, Expensive Ignorance

Inference prices keep falling. That curve isn't the one to watch. The cost that grows is the cost of not knowing, and it grows faster than token prices drop, because every stale fact has to be found and corrected by a person.

Look at where the money actually went this year. Workday's Sana now operates inside Claude, Gemini Enterprise, Microsoft Copilot, Teams, and Slack, and the headline feature isn't intelligence, it's that it respects existing permissions and approval rules. HCLTech and Anthropic announced a global partnership to deploy Claude, Claude Code, and agent tooling across large enterprises. OpenAI bought Orchestra Bio, a startup whose agents handle the operational guts of drug discovery: financial tracking, clinical trial scheduling. The team and tech went into OpenAI's life-sciences platform, Rosalind.

Notice what all three announcements have in common. None of them are really about the model getting better at generating text. They're about the model knowing where the work lives and what it's allowed to touch.

A model that doesn't know your org chart will cheerfully explain a salary band to a contractor. A model that doesn't know your approval rules will happily approve. A model that doesn't know a project was killed will keep estimating it. Permissions, org structure, project status, that's all context. It's just context with lawyers attached.

So the intuition that "smarter models will fix this" is backwards in a specific way. Better models raise the ceiling on what a well-contextualized agent can do, and raise the cost of a badly-contextualized one, because a confident wrong answer is more expensive to catch than an obviously confused one.

There's a related cost most teams never budget for: the more capable the agent, the more work it can do before anyone notices it's wrong.

A Transcript Is Not Context

Long context windows made a bad habit worse. Stuffing everything in doesn't produce understanding, it produces contradiction, and the model resolves contradictions by guessing.

Picture a Slack channel for a payments service. Four hundred messages. Buried in there: the original decision, a reversal of that decision, a partial reversal of the reversal, and a decision that was made verbally in a call and mentioned once. Two of the people who made those calls have left the company.

Now paste all four hundred messages into a context window. What does the model know? Everything, and therefore nothing. It'll pick whichever statement is most recent, most confidently worded, or most repeated, and the most repeated statement in any engineering channel is usually the wrong one.

That's the difference worth internalizing. Data is everything that was said. Context is what's true right now. A transcript is data. A decision note is context.

A usable institutional memory entry has about seven pieces: the decision, the alternatives considered, the owner, the date, the assumptions behind it, the tasks it affects, and the conversation that supports it. Two sentences of that in a task comment beats four hundred messages of raw history, every time.

The companion principle is to keep task capture and clarification apart. Capture the commitment the moment it's made, even half-formed, "check the API rate limits, Maya, before Friday." Clarify later: priority, estimate, dependencies. If capture requires a form, people won't capture. They'll remember, and then they won't.

When a task carries its own reason for existing, an agent reading it doesn't need to search six systems to act. That's not a nicer user experience. That's a cheaper answer.

Who Pays First: Freelancers and Two-Person Teams

Small teams pay the context tax in cash, not compute. Unbilled hours spent re-explaining the same client brief, and disputes over what was actually agreed, come straight out of revenue.

A freelance designer I know keeps six clients. Each has brand rules, tone constraints, an approval chain, a preferred export format, and at least one thing she's not allowed to touch. Every new tool session, every new chat, every new project file starts her at zero. She wrote a 300-word brief per client and now pastes those briefs into windows roughly forty times a month, about fifteen minutes a day of retyping her own expertise.

That's five hours a month. Twelve client-days a year. She doesn't bill for any of it.

It gets worse with commitments. Last month a client said "actually, let's prioritize the pricing page" on a Tuesday and "no, the landing page first" on Thursday. Neither landed in writing. Two weeks later, someone was annoyed about something that was decided in a hallway nobody recorded.

Enterprises can amortize this. A 200-person company spreads the re-explanation across departments and absorbs it. A two-person studio eats it whole. The tax is progressive: it hits hardest exactly where margins are thinnest.

And there's a version of it that nobody talks about, the freelancer who's the only person who knows how a thing works. Her client's project management system, her own notes, the invoice tool, and the file naming convention all live in her head. Take two weeks off and the business stops. That's institutional memory as a business-continuity problem, not a productivity one.

A single running document per client, updated when a decision changes, is the cheapest insurance in the industry. Not a nice-to-have. Infrastructure.

The Verification Line Item

AI-generated work moves labor downstream. Verification, tests, security checks, architecture fit, documentation, is the fastest-growing cost center in AI-assisted development, and most teams still budget zero for it.

Investors seem to have figured this out first. CodeRabbit, which analyzes code changes for security and quality problems, reportedly raised $143 million in a Series C in August 2026. Blacksmith, a startup focused on testing AI-generated code, reportedly raised $66 million at a $1 billion valuation. Daytona, a developer-infrastructure company, reportedly raised $60 million at a $1 billion valuation. Tenstorrent reportedly raised a $1.5 billion Series E.

Read that list twice. A huge share of the capital is going to the layer that checks work, not the layer that produces it. CodeRabbit's own positioning is telling: review, risk, quality. The thing people will pay for is confidence.

Which means your backlog is probably mislabeled. Seven kinds of work are getting smashed into one list called "to do":

• Build work, write the feature.
• Verification work, run it, test it, check it, look at the failure, check security implications.
• Decision work, pick between two architectures, confirm a scope call.
• Follow-up debt, the thing you noticed but deliberately didn't fix before release.
• Blocked work, waiting on a person, a permission, or an approval.

If only the first column shows up in your status report, AI makes your team look dramatically faster than it ships. The verification is happening either way. You're just not counting it.

Anthropic's own guidance on agent tooling makes a similar point from the other direction: agents work best when the environment is legible and the task boundaries are explicit. Ambiguity isn't free. It just moves the cost to whoever reviews the result.

Four Fixes That Cost Nothing

Most of the context tax is avoidable with habits, not tooling. Write decisions where the work already lives, keep the reason attached to the task, give "waiting" a real status, and review on a date instead of trusting a deadline.

1. Write the decision, not the discussion. Two sentences, linked to the tasks it affects: what we chose, what we rejected, and why. Don't summarize the meeting. Nobody reads summaries. Write the part that a future agent, or future you, would get wrong without it.

2. Attach context at capture time. Five fields, none of them a paragraph: action, owner, context (why it matters), next checkpoint, source. "Confirm whether the customer needs SSO in the first release; owner: founder; ask Thursday; because it changes the auth scope." Thirty seconds. Saves a day.

3. Give "waiting" a status. Blocked work is where deadlines silently die, because a backlog treats "waiting on legal" exactly the same as "not started." Those are completely different problems. One needs a nudge, the other needs an afternoon.

4. Set a latest acceptable date, not just a target. A target date is a wish. The latest acceptable date is when you must cut scope, add help, or renegotiate. Between those two numbers sits the only decision that actually matters, and picking the review date forces you to name the unknown. "Ship onboarding June 10, confidence medium, biggest unknown is the analytics migration, review May 27, fallback is shipping without historical reporting." That sentence is worth more than any Gantt chart ever drawn.

None of this requires a new model, a bigger subscription, or a migration project. It requires somebody to type a sentence while the context is still warm.

Where this goes next is predictable: as agents do more of the doing, the questions that matter become "what does it know," "who said so," and "when was that last true." Those aren't engineering questions. They're record-keeping questions, and they'll be audited, not admired. The teams that answer them cheaply will spend their money on work. Everyone else will spend it on catching up.