Entity binding failures in AI agents
For the team that owns a company master and has to defend it.
A stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, counterparty onboarding that has to survive an audit. Read the argument here, take the takeaways at the foot into your next vendor conversation, and send us a file when you want the same measurement on your own rows.
Agents · 1,673 words · every figure carries the population it was measured over
8 min read, last revised . 5 questions answered at the foot.
all 19guides →how every figure here was measured →On this page
The right tool, the wrong thing
Most agent evaluations ask two questions: did the agent choose the right tool, and did it produce valid arguments for it. The study behind the term separates a third one out. An agent can pass both and still name the wrong thing in the world, because the arguments are well-formed and point at a different person, document or account from the one the request meant. The authors formalise that as the difference between tool correctness and entity correctness, and treat a miss on the second as its own class of failure.
Their taxonomy of how it happens will be familiar to anyone who has deduplicated a customer file: name collisions, document-version ambiguity, temporal ambiguity, account collisions, near-duplicate records, cross-system references, and true ambiguity, where the request genuinely does not say which one. Company data has its own version of every one of these. A holding company and its operating company share a name stem, a registered address and often a website, and the numbered successor vehicles with a II or a III exist because the earlier ones are different legal persons.
What makes the failure expensive is that it looks like success. The transcript shows a valid call to the right tool, the tool returns without error, and the email is sent or the account updated. Nothing in the run distinguishes an action taken on the intended entity from one taken on its neighbour, so the error surfaces later, somewhere else, and usually to somebody else.
What the study measured
The evaluation is small and controlled by design. Sixty tasks across five workflows — email, calendar, documents, customer records and issue tracking — were run on five model backends with six tool-use methods, for 1,800 model–method–task runs. Each task carries an instruction, a set of candidate entities and an annotated safe behaviour, including the tasks where the safe behaviour is to ask.
The wrong-tool column is the one most agent benchmarks report, and it reads 0.0% for every method. The wrong-entity column is the one that would have sent the email. Four of the six methods are what the paper calls action-oriented baselines, and they acted on the wrong entity in 24.0% to 26.0% of runs. The two entity-aware methods, a confidence gate and entity-aware tool filtering with provenance, acted on the wrong entity in none.
| Method | Wrong tool | Wrong entity | Task success | Safe success |
|---|---|---|---|---|
| Direct | 0.0% | 26.0% | 74.0% | 74.0% |
| Semantic filter | 0.0% | 24.0% | 75.0% | 75.7% |
| CMTF only | 0.0% | 25.7% | 74.3% | 74.3% |
| Entity retrieval | 0.0% | 26.0% | 74.0% | 74.0% |
| Confidence gate | 0.0% | 0.0% | 31.7% | 40.0% |
| Entity CMTF + provenance | 0.0% | 0.0% | 26.0% | 34.3% |
The paper's figures, not Entity Bind's. CMTF is the paper's causal minimal tool filtering. Task success counts runs that chose the right tool, bound every required entity correctly and completed the action; safe success also credits a run that correctly asked or deferred on a genuinely ambiguous task. The authors describe the rates as diagnostic failure rates under the conditions they constructed, not estimates of how often deployed agents fail.
Four safeguards, and where each one lives in a resolver
The study evaluates four mechanisms for binding the right entity. Each has a counterpart in a company resolver built to refuse rather than guess, and the mapping below is ours: the paper built its mechanisms for its own tasks, and it measured nothing of Entity Bind.
Read each pair as the same job done in two places:
- Resolution preconditions. In the paper, each tool declares the entity types it needs and whether each is mandatory, and those have to be resolved before the tool runs. The resolver's version is a resolve call before the action, so the tool that acts is handed a resolved entity with a decision id rather than a name — and a payment made after a resolve has a decision id attached to it, where one made after the model decided from a website has nothing.
- Confidence-gated binding. The paper binds only when the leading candidate clears an absolute confidence threshold and a margin over the runner-up. The resolver's version is the confidence floor. A match below 0.90 confidence is unrepresentable. The record comes back as an abstention with a ranked shortlist instead. It is an acceptance contract, not calibration evidence: it says what we refuse to return, not how often an accepted match is right. Read it with the coverage it costs.
- Clarification under ambiguity. When several candidates stay plausible, the paper's gate returns a clarification request instead of executing. An abstention is that request in a form a program can route on: a ranked shortlist of up to five candidates, a typed reason for stopping, and a clarification naming the one field that would settle it.
- Provenance tracking. The paper records, for every resolved binding, the evidence for why that candidate was chosen. The decision receipt is that record: it reopens at a URL and carries the features, the thresholds the call was judged under, and the cited evidence with source URLs, so the reason an agent believed it had the right company can be checked after the context window is gone.
The trade-off, in the paper's own numbers
The entity-aware methods did not reach zero by completing every task. They deferred when the binding was unresolved, and task success fell from 74.0% to 75.0% for the action-oriented methods to 31.7% for the confidence gate and 26.0% for entity-aware filtering with provenance. The authors state the trade plainly: fewer wrong-entity actions, bought with lower direct completion.
A resolver that abstains pays the same price, and this one publishes its side of it: 70.78% (499/705) in the gate report of 2026-09-05. End-to-end resolution on the internal gated acquisition fixture with offline stub models. All 499 accepted matches were correct in this fixture. The current 99% product-target diagnostic fails; this run does not measure live model conversion or independently establish calibration.
What makes a deferral affordable is that it is cheap and specific. An abstention costs zero credits at every stage, and it names the field that would settle it, so the agent, or the person it escalates to, has a next step rather than a dead end. An agent that is unsure does not ask anybody unless the tool gives it something to ask with.
Building an agent around it
None of this needs a new model. It needs the step that picks the entity to be a separate, checkable call with a refusal the agent can branch on.
- Resolve before you act, and pass the resolved entity id to the tool that acts, never the name the request arrived with.
- Branch on the decision rather than on a score: a match to act on, an abstention to escalate or enrich, or a non-match.
- Treat an abstention as the clarification request it is. Ask the person, or fetch the field it names from the contract or the invoice, then resolve again.
- Keep the decision id with the action, so a reviewer can reopen why the agent believed it had the right company.
- Count completed tasks and wrong-entity actions separately. A single success rate hides exactly the trade the study measured.
What this does not show
The study's rates come from 60 constructed tasks with controlled environments and entity stores, and its authors are explicit that they are diagnostic failure rates under those conditions rather than a measure of how often deployed agents act on the wrong entity. Quote them with that attached.
Nothing in the study measured Entity Bind, and no figure on this page from the paper should be read as one of ours. Entity Bind's own figures, each with the population it was measured over, are on the accuracy page.
Most of the paper's failure classes sit outside what a company resolver does. A wrong email recipient, a wrong calendar event or a stale document version is not a company identity problem. A wrong customer account or supplier record is, where the customer or supplier is a company, and that is the part this resolver addresses: companies and organisations only, never people.
What to take away
- An entity binding failure is the right tool acting on the wrong real-world entity. A metric that only checks tool choice cannot see it.
- In a controlled study of 1,800 runs — 60 tasks, five model backends, six methods — every method chose the right tool, and the four action-oriented methods still acted on the wrong entity in 24.0% to 26.0% of runs.
- The two methods that checked the binding first made no wrong-entity actions in that setting, and completed 31.7% and 26.0% of tasks against 74.0% to 75.0%, because they deferred under ambiguity.
- A resolver supplies the same four safeguards: resolution before the action, a confidence floor, an abstention that names the missing field, and a decision receipt. The floor here is 0.90. It is an acceptance contract, not calibration evidence: it says what we refuse to return, not how often an accepted match is right. Read it with the coverage it costs.
- The rates are the paper's, from a constructed setting. They are neither a prevalence estimate nor a measurement of Entity Bind.
Those takeaways hold for anybody's data. The number for yours comes out of your own rows — a few hundred of them, with the matches you already have beside them.
Start freePrefer to put it to a person? Talk to us instead →
Where this article is applied
1 pages name this method as the working behind what they claim.
Is this your file?
The article is for anyone. The measurement behind it is worth commissioning for one kind of team.
Teams with a company master they have to defend — an MDM or stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, or counterparty onboarding that has to survive an audit.
Buying breadth as the product: we hold a fraction of an incumbent’s records and are not building a global company graph. Entity Bind resolves companies and organisations only. It does not resolve people, vessels, bank-account ownership, sanctions exposure, emissions or geolocation, and its results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Sources
The papers, regulators and registers this article quotes, each with the caveat that travels with it.
- Suresh Babu and Indukuri (2026), “Entity Binding Failures in Tool-Augmented Agents”, arXiv:2606.30531
An arXiv preprint, and by its authors' own description a controlled diagnostic evaluation: 60 tasks over email, calendar, documents, customer records and issue tracking. Its rates are diagnostic failure rates under the constructed conditions, not estimates of deployment-wide prevalence.
Internal fixtures only. The historical 2026-08-29 per-tier recording covers 317 GB companies with identifier-rich arrivals seeding the catalogue first; the context tier includes country and registered-town prose. Its legacy replay has no full-prompt-verified calls and cannot certify current calibration. The September 5 CI reports cover 1,176 labelled pairs and 705 records over 250 companies using offline stubs. These are not independent customer holdouts or live production validation; the 99% minimum-input product target remains unmet. Scope is company and organisation identity only — not people, vessels, bank-account ownership, sanctions or emissions. Results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Figures read off the product repository on 2026-09-05. The per-tier rates come from the live-adjudicated 317-company recording of 2026-08-29; the gate figures come from the stub-adjudicated continuous-integration run and are a floor rather than a ceiling. The two are never averaged. Entity Bind holds no SOC 2, no ISO 27001 and no other certification, and claims none. Every figure, with the population it was measured over and the method behind both recordings, is set out on the accuracy page. The retention, erasure, tenancy and sub-processor detail a security review asks for is under Trust and security, with where assurance stands: no external validation of the accuracy figures and no customer holdout.
The same measurement, on your own rows.
Create your workspace, verify your email and run your first company file. The Free plan needs no card. Create API keys in your workspace when you need an integration.
300 credits. No card.
For a guided review of your file, explore the Merge Safety Sprint →
The questions this article answers
What is an entity binding failure?
An AI agent choosing the right tool and acting on the wrong real-world entity: emailing the wrong person, replying in the wrong thread, updating the wrong account. The term is the one a 2026 study of tool-augmented agents uses, arXiv:2606.30531, which separates tool correctness from entity correctness and treats a miss on the second as a distinct reliability and safety problem.
How often do AI agents act on the wrong entity?
The study behind the term does not claim a deployment-wide figure. It measured 24.0% to 26.0% of runs for four action-oriented methods, across 60 tasks, five model backends and six methods, with 0.0% wrong-tool errors throughout. Its authors describe those as diagnostic failure rates under conditions they constructed, not an estimate of prevalence, so quote the population with the number.
Does checking the entity first make an agent less useful?
It makes it complete fewer tasks directly, and the study measured by how much: 31.7% and 26.0% task success for the two entity-aware methods against 74.0% to 75.0% for the action-oriented ones, with zero wrong-entity actions in exchange. Whether that is less useful depends on what a wrong action costs. A deferral that is free and names the field that would settle it is a next step; an email sent to the wrong customer cannot be unsent.
Is the confidence floor the same as the paper's confidence gate?
They do the same job and are not the same mechanism. The paper's gate requires both an absolute confidence and a margin over the second candidate. Entity Bind's floor is 0.90. A match below 0.90 confidence is unrepresentable. The record comes back as an abstention with a ranked shortlist instead. It is an acceptance contract, not calibration evidence: it says what we refuse to return, not how often an accepted match is right. Read it with the coverage it costs.
Which entity binding failures does Entity Bind address?
Only those where the entity is a company or an organisation: the wrong customer account or supplier record, where the customer or supplier is a company. A wrong email recipient, calendar event or document version is outside its scope, and it never resolves people. What it gives an agent is a resolve call that returns a decision, an abstention with a shortlist and the field that would settle it, or a non-match, with a receipt that reopens at a URL.