Thirty Seconds, One Attempt: Tool Calls to Business Systems

The request that has an audience

Mid-sentence, the model decides it needs something it does not know: whether this caller has an open order, whether Thursday at ten is free, what the balance is. It emits a tool call, which on our platform becomes a signed HTTP POST to an endpoint you own, carrying the call context and the arguments the model chose.

From that moment a clock runs that has nothing to do with your infrastructure. It is the clock in the caller's head. Two seconds of silence on a phone line is a pause. Five is a problem. Eight and people say hello to check whether the line is still there. The technical timeout matters far less than that, and on our platform a synchronous tool call waits up to thirty seconds. The thirty seconds is the ceiling before the system gives up. It is not a budget you should plan to spend.

caller: "can you check my last invoice"   t+0.0s
  [llm]   tool call: lookup_invoice           t+0.9s
  [http]  POST https://you.example/tools      t+0.9s
  [say]   "one moment, let me look"           t+1.1s   <-- filler, not a stall
  [erp]   token read from cache               t+1.1s
  [erp]   filtered query, one record          t+1.2s
  [http]  200 · 412 bytes                     t+3.4s
  [llm]   answer generated                    t+3.9s
  agent:  "that one was paid on the fourth"   t+4.2s

  tool budget used                            2.5s of 30s
  attempts                                    1 of 1

The call above took two and a half seconds inside a thirty second allowance, and the caller heard a short filler line rather than dead air. That is the shape you are aiming for. Everything else in this article is about the ways it goes wrong.

Why one attempt and not three

Retrying a failed request is such standard practice that leaving it out looks like an oversight. On a live call it is the opposite. A retry that succeeds four seconds later returns an answer to a question the conversation has already moved past, and the agent either says something irrelevant or has to be told to discard it. Worse, the caller has spent those four seconds listening to nothing.

So on our platform a synchronous tool call is attempted exactly once. Event webhooks, which fire after the fact and have no audience, behave the other way around: a shorter ten second timeout and up to three attempts, because there nobody is waiting and delivery matters more than speed.

There is a second reason, and it comes from the systems on the other end rather than from us. Exact Online allows no more than ten errors per API key, per user, per company, per endpoint, per hour, and blocks the key once you exceed it. The block clears by itself an hour later. An agent that retries three times on every failure reaches that ceiling during precisely the incident you most want to keep short, and turns a bad minute into a bad hour.

Reads block, writes should not

Almost every integration splits cleanly into two halves that deserve opposite designs. The read half answers a question the agent needs before it can speak, so it has to block. The write half records something that happened, and nobody has ever stayed on the line to hear a CRM confirm it.

1. 01

   Reads on the call path, filtered and narrow

   One record, selected by an identifier you already have, returning the four fields the agent might say out loud. Anything that needs a broad query is not a call-path read, and pretending otherwise is how a lookup becomes a nine second silence.
2. 02

   Writes queued for after the call

   The outcome, the transcript summary, the created ticket. Accept the tool call, put the work on a queue, return immediately. The agent gets on with the conversation and your system gets the record a few seconds later.
3. 03

   The exception that proves it

   A write the caller needs confirmed out loud, such as a booked appointment, has to block. That one is worth the wait, because the caller is waiting for exactly that answer and knows it.

The mistake to avoid is treating the split as a performance optimisation to apply later. It is a design decision, and reversing it once the agent is live means changing what the agent says, not just how fast it says it.

What the agent says while it waits

The single largest improvement available here is not technical. It is that the agent should say something before the lookup starts, not after it fails. A caller who hears one moment, let me check that will wait comfortably for four or five seconds. The same caller, hearing nothing at all, starts wondering after two.

This is harder than it sounds with speech-to-speech models, which do not naturally narrate their own tool use, and it is one of the things that has to be written into the instruction rather than hoped for. We cover the prompting side of that in the Gemini Live prompting guide, and the timing side in the latency budget of a phone call.

A slow lookup with a filler line is a conversation. A fast lookup with dead air is a fault.

The caller will ask twice

People repeat themselves on the phone. They rephrase when they think they were not understood, they say the date again to be sure, and if the line goes quiet they will often just ask the whole question a second time. Each of those can produce another tool call with the same intent, and if your endpoint writes on every call you now have two appointments, two tickets or two orders.

The fix is ordinary and needs to be in place from the first write rather than added after the first duplicate. Every tool call carries an identifier for the call and one for the tool invocation. Derive a key from the call plus the action and the significant arguments, store it, and let a repeat land on the same key and return the original result. Then the second attempt is free and the caller hears the same confirmation twice, which is exactly what they wanted.

Return an answer, not a record

The most common cause of a strange sounding agent is an endpoint that returns what the API returned. A CRM record has forty fields, three of them nested, and a model handed all of that will pick something to say that you did not expect. It might read out an internal identifier. It might mention a field a caller should not hear.

Return the smallest thing that answers the question, already shaped for speech. Not the invoice object, but the fact that invoice 2026-0412 was paid on the fourth of August. Dates in words rather than in ISO. Amounts rounded the way a person would say them. Nothing the caller is not entitled to hear, because a model given a field will eventually find a reason to use it.

Failing out loud

Something in this chain will be down at some point: the ERP, the network in between, or your own endpoint during a deploy. The question that decides whether that becomes a complaint is what the agent does about it.

It says so. The tool call times out, the agent is told the lookup failed, and it tells the caller it cannot check right now and offers a callback or a transfer to someone who can. What it must never do is fill the gap with a plausible answer, and this is not a per-integration preference. It is the same fail closed position that governs what happens when a transfer fails: a refusal the caller can act on beats a confident answer that turns out to be invented.

The other half of failing well is that you can see it afterwards. Every tool call is logged with its status, its latency and its response, so a caller complaining that the agent did not know about their order becomes a question you can answer from the record rather than a story you have to reconstruct. That is the same reasoning as reading the SIP logs, one layer up.