Saving Tokens as a Transportation Problem

Measure the agent’s entire trajectory

I would measure not the tokens of a single request, but the expected cost of the agent’s full trajectory. In the original discussion, user 1497575255 compared token minimization to a transportation problem: the result must travel from point A to point B while consuming the fewest resources. That is more accurate than simply saying, “let’s shorten the prompt.”

For every step, the input and output cost can be known or estimated. Input includes instructions, history, tool schemas, and retrieved documents. Output depends on the task type, selected model, generation budget, and whether the agent must retry the call.

At pipeline level, there are more variables:

• which model should handle planning, execution, and verification;
• which context fragments should be retained or compressed;
• how many iterations and retries are allowed;
• when to call a tool and when to end a branch.

The objective then minimizes more than token count: it minimizes total cost under quality, latency, and reliability constraints. Assigning a task to one of several models becomes an integer-selection problem. When request flow can be distributed across models, configurations, or regions, the transportation analogy becomes almost literal.

Context selection is a separate subproblem. Each fragment has a size and expected utility, while the total context window is budget-constrained. Structurally, this resembles a knapsack problem combined with model routing in one optimization scheme.

Input length can usually be calculated before sending a request. Output is harder: it requires forecasts by task class and model, while agent workflows must also account for the probability of extra steps. As of November 6, 2026, this is not news about a specific release, but a useful engineering framing of the problem.

Savings do not start with the cheapest model

The biggest gains come from managing the agent’s decisions, not mechanically replacing a strong model with a weaker one. An expensive call can be cheaper overall if it prevents retries, unnecessary search, or a long reasoning branch.

A practical architecture has two parts. A predictor estimates tokens, quality, latency, and risk for possible actions. A controller chooses the least expensive option that meets defined constraints, then recalculates after each step using the remaining budget.

The weakness is clear: an error in quality prediction can cost more than the tokens saved. I would therefore monitor repeated calls, growing conversation history, and passing full tool outputs most closely. Those are the places where agent pipelines often burn budget unnoticed.

In such a system, tokens are not merely a post-execution counter; they are a planning resource. One question remains: how accurately can the router predict the cost of failure before a cheap path becomes an expensive detour?

We previously examined how Claude Opus 4.6 configurations, extended reasoning, and context length affect request costs. This complements the view of token optimization as allocating a limited budget across parts of the context.