Should I bill an AI agent per token or per turn?

Per turn is far easier to sell, because a customer can predict it, and it is the model almost every assistant product now uses. The cost of that simplicity is that you absorb the variance inside a turn. That only works if you know the real distribution of a turn's token cost, not the one your caps imply. Our flat charge is one coefficient per model run, and when we measured 1,000 production turns the median round carried 19,607 prompt tokens against a priced worst case of 16,000.

How do I set a flat per-turn price that cannot go underwater?

Derive the floor from measured production rounds rather than from the limits in your code, and derive it from a high percentile rather than an average. Then bound every prompt segment a customer can grow. In our case the conversation history was capped at 16,000 tokens and the tool schemas were not capped at all, so a customer who bound 103 tools set the price of their own turns.

Why does an agent turn cost so much more than a single chat completion?

A turn is several requests. The model proposes a tool call, you execute it, you send the result back, and it proposes the next one. Every round re-sends the system prompt, the tool schemas and the accumulated tool results. In our data the median turn is one round and the 90th percentile is three, with a maximum of nine, so the heavy turns pay for the same prompt up to nine times.

What should I measure before pricing an AI feature?

Prompt tokens per round, split by segment, from production traffic, plus the provider's reported token count rather than your own estimate. Our character-based estimator read about one fifth of what the provider billed at the median, which is enough of a gap to make a pricing floor meaningless.

Metered billing for AI agents: the worst case we priced for was below our median round

A flat charge per AI reply needs a worst-case round cost underneath it, and ours was 16,000 input plus 1,024 output tokens, taken from our own history cap. Then we measured 1,000 production turns: median round 19,607 prompt tokens, 90th percentile 70,974, heaviest 273,187, with tool schemas rather than conversation history doing the damage. Here is the arithmetic, the measurement, and what a per-turn price has to bound before it is honest.

We publish our billing rule in plain language: credits per reply equals the model coefficient times the number of model runs. A reply on a 1-coefficient model costs 1 credit, and our cheapest credit pack prices a credit at $0.02, since $10 buys 500 of them. No tokens, no metered overage, nothing a customer has to model in a spreadsheet.

Underneath that rule there is an assumption nobody sees. A flat charge per round is only safe if the worst round costs less than the charge, so somewhere in the code there has to be a number for "the worst round". Ours was 16,000 input tokens and 1,024 output tokens. Last week I measured 1,000 real turns against it. The median round carried 19,607 prompt tokens and the heaviest carried 273,187.

Why a flat charge needs a worst case at all

The guard is four lines of arithmetic. For every model in the catalogue, the coefficient times $0.02 has to cover the worst-case round cost at that model's published token prices:

export const CREDIT_USD_MICROS = 20_000;
export const WORST_CASE_INPUT_TOKENS = 16_000;
export const WORST_CASE_OUTPUT_TOKENS = 1_024;

export function worstCaseRoundMicros(p) {
  return Math.ceil(
    (WORST_CASE_INPUT_TOKENS * p.promptPerMtokUsdMicros +
      WORST_CASE_OUTPUT_TOKENS * p.completionPerMtokUsdMicros) / 1_000_000,
  );
}

That function set every coefficient we ship. On Gemini 3 Flash, at $0.50 per million input tokens and $3.00 per million output, a worst-case round works out to $0.0111, so 1 credit covers it with room to spare. Claude Haiku 4.5 comes out at $0.0211 and gets 2 credits. Claude Opus 4.8 comes out at $0.1056 and gets 6. The published coefficient table is that calculation, rounded up to whole credits.

The arithmetic is right. The input to it was a guess dressed as a constant.

The assumption described the one segment we control

16,000 did not come from production. It came from our own history cap, which is a real constant in the request path:

export const HISTORY_INPUT_TOKEN_CAP = 16_000;
export const TOOL_RESULT_TURN_BUDGET_CHARS = 24_000;
export const DEFAULT_OUTPUT_TOKENS = 3072;
export const HARD_OUTPUT_CEILING = 8192;

Conversation history plus the visitor's message gets trimmed to 16,000 tokens, and a message over the cap is rejected with an explicit error. So the pricing floor was built from the only segment of the prompt that already had a ceiling, and the prompt has five segments. When we started recording them per turn, the breakdown looked like this:

system_tokens, the assistant's instructions, which we write.
history_tokens, capped at 16,000.
knowledge_tokens, the retrieved documentation for this message.
tool_results_tokens, whatever the customer's API returned this turn.
tools_tokens, every bound tool's name, description and full JSON Schema, re-sent on every round.

Four of those five are sized by the customer's deployment, not by us. The one we capped turned out to be the smallest: across 634 recorded turns, history averaged 595 tokens while tool schemas averaged 2,051 and ran to 27,151 on the heaviest turn, where 103 tools were bound.

Our own caps already broke the floor

Before any customer data, the two constants above are already inconsistent. The floor assumes 1,024 output tokens. The runtime defaults to 3,072 and lets a widget be configured up to 8,192, which is the number a customer actually gets if they ask for longer answers.

At Gemini 3 Flash prices, 8,192 output tokens cost $0.0246 on their own. That is more than the $0.02 that 1 credit covers, before a single input token is counted. Add a full 16,000 token input and the round costs $0.0326, which is 1.6 times the charge. The cheapest model in the catalogue, inside limits we wrote ourselves, can produce a round that the price does not cover. Nobody had to misuse anything.

A guard that cannot fail the request is a log line

There are two copies of this check. The one in the test suite throws, so a bad coefficient cannot be committed. The one in the request path writes [pricing] credit_coefficient_violation to the console and returns the pricing anyway, because resolution must never break a live chat. That was the right call for availability and the wrong call for billing: the violation it was built to catch had to be found by querying the usage table by hand.

What production actually sent

The numbers below come from 1,000 widget chat turns between 9 September and 10 October 2026, all on one model, across two workspaces, one of which is a real customer deployment generated from a GraphQL API. Prompt tokens are the provider's reported count for the turn, divided by the number of rounds in that turn.

Median round: 19,607 prompt tokens, already above the 16,000 we priced for.
90th percentile: 70,974. 99th percentile: 162,657. Heaviest: 273,187.
526 of the 1,000 turns had a per-round average above the priced worst case.
Rounds per turn: 1 at the median, 3 at the 90th percentile, 9 at the maximum.

The week-by-week view says something sharper than the aggregate. In the first week of that window the median round was 1,520 tokens, because the deployment was still small. Once the full tool set went live the median round moved to between 27,995 and 43,184 and stayed there. We changed neither the model nor the price. A customer connected their API, and the cost of an average round on that widget went up by more than an order of magnitude.

This is what makes per-turn pricing different from per-token pricing in a way that is easy to miss. The variance is not driven by how much a visitor types. It is driven by how much surface the customer connected, and that number arrives without warning after they have already agreed to your price.

Our own token estimate was off by five times

The per-segment breakdown is computed locally, and the estimator is the usual one: character count divided by four, doubled when the text has CJK or emoji in it. Comparing it to the provider's reported count on the same turns, the provider billed about 4.8 times the estimate at the median and 11 times at the 90th percentile.

Dense JSON is the reason. A tool schema is mostly punctuation, quotes and short identifiers, and a real tokenizer produces far more tokens per character for that than for prose. Which means the segment that dominates a heavy prompt is also the segment our estimator was worst at reading. The breakdown is still useful for proportions, but any number that sets a price has to come from the provider's count.

What we changed

The response was not to re-price. It was to put a ceiling on the segments that did not have one, because a flat per-round charge is a promise that the round is bounded.

Tool schemas now go through per-turn retrieval: above 24 bound tools, the top 20 for this message survive into the prompt, which cut schema tokens by about 75% on the 84-tool deployment. Tool results go through a single 24,000 character evidence budget for the whole turn, rather than a per-result cap that ten results could walk straight past.

What is still open is the floor itself. A constant that was never measured should not be the thing standing between a published price and the provider invoice. The honest version reads a high percentile of the last 30 days of real rounds per model, recomputes the required coefficient, and makes a violation visible somewhere a person looks rather than in a Worker log.

If you are metering an agent, not a chat

Record prompt tokens per round, split by segment, from the first day of traffic. You cannot reconstruct this later from an invoice.
Set the floor from the provider's reported tokens at a high percentile of production rounds. An average hides the turns that cost you money, and your own estimator is probably reading a fraction of the truth.
List every prompt segment and ask who sizes it. Any segment a customer can grow needs its own ceiling before a flat price is safe.
Re-run the check when a customer's deployment changes, not when your price changes. Ours moved by more than an order of magnitude because someone connected an API, which is exactly the day nobody is looking at billing.

The number that matters is the one you measured

A pricing model built on the limits in your own code describes the product you imagined. The distribution in your usage table describes the one you shipped. We had both, written eight weeks apart, and only one of them was true.

Every widget hosted on Command+K is metered this way, in credits per reply, with the per-turn breakdown recorded for every turn. The numbers in this post came out of that table, which is the only reason I could check the assumption instead of believing it.

FAQ

Should I bill an AI agent per token or per turn?: Per turn is far easier to sell, because a customer can predict it, and it is the model almost every assistant product now uses. The cost of that simplicity is that you absorb the variance inside a turn. That only works if you know the real distribution of a turn's token cost, not the one your caps imply. Our flat charge is one coefficient per model run, and when we measured 1,000 production turns the median round carried 19,607 prompt tokens against a priced worst case of 16,000.
How do I set a flat per-turn price that cannot go underwater?: Derive the floor from measured production rounds rather than from the limits in your code, and derive it from a high percentile rather than an average. Then bound every prompt segment a customer can grow. In our case the conversation history was capped at 16,000 tokens and the tool schemas were not capped at all, so a customer who bound 103 tools set the price of their own turns.
Why does an agent turn cost so much more than a single chat completion?: A turn is several requests. The model proposes a tool call, you execute it, you send the result back, and it proposes the next one. Every round re-sends the system prompt, the tool schemas and the accumulated tool results. In our data the median turn is one round and the 90th percentile is three, with a maximum of nine, so the heavy turns pay for the same prompt up to nine times.
What should I measure before pricing an AI feature?: Prompt tokens per round, split by segment, from production traffic, plus the provider's reported token count rather than your own estimate. Our character-based estimator read about one fifth of what the provider billed at the median, which is enough of a gap to make a pricing floor meaningless.