The assistant said it created the draft. It never called the tool.

A customer opened our widget on 19 September, asked it to create a draft post, and got this back:

I have created the draft post titled "AnKit AI QA draft 3" in the
Acme project. The post has been saved as a draft with the body
"Third draft, round 7."

No draft existed. No tool ran in that turn. The assistant had the project name, the title and the body in context, and it wrote the sentence that usually follows.

This is the worst failure our product can have. A wrong answer is an annoyance. An answer that reports a write nobody performed is a lie the user acts on: they close the tab, the draft is not there, and they stop trusting every other answer the assistant has given them. It also cannot be found in logs, because nothing failed. Nothing returns an error and no isError flag is set anywhere. The turn looks perfect.

We shipped three fixes for it. Two are about why the model was asked to answer with no tools at all, and the third is a check that runs on the way out, after we accepted that the first two will never be complete.

Cause 1: a two-word command classified as small talk

We route every turn before we call the model. Small talk gets no tools and no retrieval, because attaching 40 tool definitions to answer "thanks" is a waste of tokens and a source of unwanted tool calls. The classifier had a cheap rule at the bottom:

const words = t.split(/\s+/).filter(Boolean);
if (words.length < 3) return true;   // -> chitchat

The user had written Evet, oluştur., which is Turkish for "Yes, create it." Two words. Classified as small talk, so the turn was planned with no tool scope. The model received the instruction and its own previous offer to create the draft, and no ability to do anything about either.

The word-count intuition comes from English, where two words rarely form a command. Turkish is agglutinative: the imperative and the object can both live inside one inflected word. Our fix was not to raise the threshold, it was to check for a command verb before the count decides anything:

const COMMAND_WORD_RE =
  /(?<![\p{L}\p{N}])(?:create|make|save|update|edit|delete|remove|add|send|post|publish|cancel|rename|assign|approve|continue|proceed|retry|oluştur|olustur|yarat|kaydet|güncelle|guncelle|düzenle|duzenle|sil|kaldır|kaldir|ekle|gönder|gonder|yayınla|yayinla|iptal|taşı|tasi|değiştir|degistir|paylaş|paylas|onayla|ata|devam|başlat|baslat|durdur|dene)(?![\p{L}\p{N}])/iu;

if (COMMAND_WORD_RE.test(t)) return false;   // never chitchat
const words = t.split(/\s+/).filter(Boolean);
if (words.length < 3) return true;

If you have a length-based small-talk rule anywhere in your routing, this is the bug you have too, and you will only see it in the languages you do not test in.

Cause 2: a bare "yes" inherits nothing

The second path into the same hole is a bare confirmation. The assistant offers to do something, the user says yes, and the router sees a four-letter message carrying no subject and no object. Every signal it looks for is in the previous turn, not this one.

So the decision cannot be made from the user's message alone. It needs the assistant's last message: if that message was an offer to write, then "yes" is a write turn.

if (isAffirmativeContinuation(text, history)) {
  const continuesWrite =
    looksLikeWriteRequest(text) ||
    looksLikeWriteProposal(lastAssistantText(history));
  if (caps.hasMcpWrite && continuesWrite) {
    return planForIntent("action", mode, caps, 0.8, "rule");
  }
  return planForIntent("mixed", mode, caps, 0.75, "rule");
}

looksLikeWriteProposal matches the assistant's own offer grammar: "shall I create", "would you like me to publish", and the Turkish first-person-optative forms that carry the same meaning in one word (oluşturayım, kaydedeyim). Affirmation plus a trailing command ("okay, create it") counts too, capped at five words so a long sentence that merely starts with "sure" still goes through normal classification.

Cause 3: assume the router will be wrong anyway

Our system prompt has told the model for months never to claim a completed action without a tool result. The prompt rule is still there and it is still worth having. It did not prevent the sentence at the top of this post, and no wording of it will, because a prompt shifts a probability and this needs a decision.

The decision is cheap, because we already know the one fact that settles it: how many tools executed in this turn. If that number is zero, no write happened. Nothing about the model's confidence changes that.

export function shouldGuardWriteClaim(input: {
  text: string;
  executedToolCalls: number;
  approvalPending: boolean;
  writeContext: boolean;
}): boolean {
  if (!input.writeContext) return false;
  if (input.approvalPending) return false;
  if (input.executedToolCalls > 0) return false;
  return looksLikeWriteClaim(input.text);
}

Three of those four conditions exist to keep the guard quiet. A read-only turn is not checked at all. A turn that is waiting on an approval card is allowed to describe the pending action. A turn where any tool ran is out of scope, because the claim might be true and this check has no business adjudicating it. What is left is narrow: a write turn, no tool, and a sentence that says the write is done.

When the guard fires we replace the reply with the honest version:

I haven't made that change yet — I need to run the action first.
Tell me to go ahead, or check the details above.

We also push a nudge into the next model call telling it that no tool has executed and that it should either call the tool or ask for the missing detail. The replacement text is written so it never matches the claim detector itself, which is worth a test of its own.

Detecting a claim without flagging a plan

The detector is the part most likely to go wrong, because the difference between a claim and a plan is grammar, not vocabulary. Both sentences contain "created":

• Claim: "I have created the draft." The guard must fire.
• Plan: "The draft will be created once you approve." The guard must stay silent, or the assistant can never describe what it is about to do.

A false positive here is expensive. It turns a correct, useful sentence into "I haven't made that change yet", which is itself a lie in the other direction. So the detector matches completed-action grammar and then looks backwards from the match for anything that cancels it:

const CLAIM_NEGATION_RE =
  /\b(?:not|never|no|n't|haven't|hasn't|didn't|cannot|can't|won't|unable|yet to|need(?:s)? to|to be|will be|would be|should be|can be|may be|must be|before|once|after|if|when|until|unless|without|how to|whether|able to|going to|about to|ready to|want(?:ed)? to|i'll|i will|i can|i could|so that|in order to)\s*[^.\n]{0,24}$/i;

The window matters. We look at the 48 characters before the match and require the canceling word to be within 24 characters of it, so "I can create the draft for you" is a plan while a paragraph that happens to contain "not" three sentences earlier is still a claim.

The Turkish side needed a different shape entirely. Our first version was a list of finished forms, oluşturuldu among them. Production produced oluşturulmuştur, the same verb in the passive perfect, and the list missed it. In an agglutinative language the forms are open-ended, so the pattern has to be stem plus suffix, with the suffixes that mean not done excluded:

const TR_NOT_DONE   = "(?:m[ae](?:d[ıiuü]|m[ıiuü]ş|z)|[ae]cak|[ae]cek|abil|ebil)";
const TR_DONE_SUFFIX = "(?:d[ıiuü]|t[ıiuü]|m[ıiuü]ş)(?:t[ıiuü]r)?";

// oluşturuldu / oluşturulmuştur -> claim
// oluşturulamadı (-madı)        -> failure, not a claim
// oluşturacağım  (-acak)        -> intention, not a claim
// oluşturabilirim (-abil)       -> capability, not a claim

Negation, future and the potential mood are three separate exclusions, and each one is a sentence a support assistant says constantly. Getting any of them wrong replaces a correct answer with a denial.

The tests are the specification

This kind of check only stays correct if the boundary is written down as examples. Ours is a table of strings that must fire and strings that must not, in both languages, including the exact sentence from production:

for (const t of [
  PROD_CLAIM,
  "I've created the draft for you.",
  "Your changes were saved.",
  "Done! The draft is created in Acme.",
  "Taslak gönderiyi oluşturdum.",
  "'QA Türkçe 19 Eyl' başlıklı taslak gönderiniz başarıyla oluşturulmuştur.",
]) expect(looksLikeWriteClaim(t), t).toBe(true);

for (const t of [
  "I haven't created the draft yet — which project should it go in?",
  "The draft will be created once you approve the action.",
  "To publish a post, open the editor and click Publish.",
  "This needs your approval before I can run it: create_post.",
  "Taslak oluşturulamadı, çünkü projenin varsayılan dili en.",
  "Onaylarsanız taslağı oluşturacağım.",
]) expect(looksLikeWriteClaim(t), t).toBe(false);

Every line in the false list is a real sentence the assistant needs to be able to say. When we add a language, that is where the work goes first.

What to copy if you ship an agent that writes

• Count the tool calls that actually executed in the turn and keep that number next to the outgoing text. Everything here depends on having it.
• Never let a length or small-talk heuristic remove tool access. Check for command verbs first, in every language your users write in.
• Route a bare "yes" using the assistant's previous message. On its own it carries no intent at all.
• Treat the prompt rule as a nudge and the outgoing check as the guarantee. Keep both.
• Make the honest replacement text un-matchable by your own detector, and test that it is.
• Detect claims by stem and suffix, not by a list of finished phrases, as soon as you support a language that inflects.

The guard sits in the same family as the isError bugs we wrote about earlier: the tool ran and failed there, no tool ran at all here, and in both cases the user was told the write succeeded. Neither produced a log line. If you are deciding which actions to expose at all, assume both failures will happen to you and build the check before the first customer finds it.

Every assistant hosted on Command+K gets this guard at the gateway, so a deployment inherits it without writing any of it.