--- title: "Precision, recall and abstention" summary: "Entity resolution is a three-way trade, not a two-way one. Why abstention is a design choice rather than a failure, what an abstention has to carry to be useful, and how to measure a system that declines." canonical_url: https://entitybind.com/guides/precision-recall-and-abstention last_updated: 2026-10-12 copy: full --- # Precision, recall and abstention > Entity resolution is a three-way trade, not a two-way one. Why abstention is a design choice rather than a failure, what an abstention has to carry to be useful, and how to measure a system that declines. Canonical page: https://entitybind.com/guides/precision-recall-and-abstention Last updated: 2026-10-12 For the team that owns a company master and has to defend it. The precision-recall trade-off is usually described as a two-way choice, and in entity resolution that description is wrong. There is a third output — declining to decide — and it changes the shape of the problem entirely. Once a system can abstain, precision and coverage stop being locked together: you can hold precision arbitrarily high and pay for it in coverage rather than in wrong merges. Abstention is therefore a design choice about where the cost lands, not an admission that the model is weak. - Reading time: 8 min read - Sections: 8 - Questions answered: 5 - Key takeaways: 5 The same measurement, on your rows: A first file is a few hundred of your own rows with the matches you already have beside them. The numbers in this article come back measured on your data, and we write nothing to your systems. [Start free →](https://app.entitybind.com/app?signup=1) Self-service. Create your workspace online and verify your email to start matching company records. Free starts with 300 credits and no card. Paid monthly plans are available in workspace Billing; VAT is added where applicable. Usage is tracked without automatic overage charges. [Trust and security](https://entitybind.com/trust) A stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, counterparty onboarding that has to survive an audit. Read the argument here, take the takeaways at the foot into your next vendor conversation, and send us a file when you want the same measurement on your own rows. Measurement · 1,856 words · every figure carries the population it was measured over By [James Rooney](https://entitybind.com/about), founder of Entity Bind · Last revised 12 October 2026 ## Two errors with very different prices Start with who pays. A false merge joins two different companies into one entity. Every aggregate over that entity is now wrong, the error is invisible until someone reconciles by hand, and unwinding it means rewriting downstream state that has already been acted on. In a credit or compliance context it converts into damages. A false split leaves the same company represented twice. Someone eventually notices a duplicate supplier or a double-counted exposure, and the fix is a later pass with better evidence. It costs money and it does not cost trust in the same way. Because the two prices differ by roughly an order of magnitude, the correct operating point is not the one that maximises F1. It is the one that holds precision at the level the downstream process actually requires and takes the coverage hit. Anyone who cannot state which of these two errors their configuration prefers has not chosen an operating point; they have accepted a default. The three outcomes, and who bears each cost: | Outcome | What happened | Who pays | How it is found | | --- | --- | --- | --- | | Correct decision | Records joined to the right entity, or correctly kept apart. | Nobody. | It is not found. It is the baseline. | | False merge (precision failure) | Two different companies share one identity. | Everything downstream that aggregates over the entity. | Manual reconciliation, usually months later, usually by accident. | | False split (recall failure) | One company appears as two entities. | Operations, in duplicate work and double counting. | Duplicate reports, and a later pass with more evidence. | | Abstention | The system declined and said so. | Whoever works the review queue. | Immediately. It is in the response. | Rows two and three are where the asymmetry lives, and it is why a well-built resolver prefers under-merging. Row four moves a cost from an invisible place to a visible one. ## What abstention buys A system with only two outputs has to place every borderline pair on one side of a threshold, and borderline pairs are where the errors are. Adding a third output means the threshold becomes two thresholds and the ambiguous band between them stops producing errors at all — it produces work instead. Cutting the score line at two points rather than one makes the band somebody has to look at as small as it can be for any fixed pair of error rates. That is the practical content of [Fellegi and Sunter's 1969 result](https://doi.org/10.1080/01621459.1969.10501049): once you have decided how much precision you need, the best thing to do with what is left over is to send the smallest possible slice to a human, and that slice is defined by two cut-offs rather than one. The consequence is that a resolver's headline precision figure becomes a statement about a chosen operating point rather than about the model. This is why an abstaining system's precision must always be published with its automation rate: 742 of 1,176 pairs were decided automatically in the September 5 saved gate report. This measures automation, separately from accepted-merge precision. ## Choosing the operating point is a business decision Because the three outcomes trade against each other, somebody has to decide where the two thresholds sit, and that decision belongs to whoever owns the downstream process rather than to whoever owns the model. The input to it is not a metric. It is the relative cost of the two errors in the specific process the output feeds. The arithmetic is worth doing explicitly, even roughly. If a wrong merge in a supplier master costs a day of finance time to find and unwind, and a duplicate supplier costs an hour of duplicate handling, the thresholds should be nowhere near symmetric. If the output decides which legal entity a payment is routed to, the cost of a wrong merge is not measured in hours at all, and the correct operating point is one where the automation rate looks disappointing on a slide. There is a methodological trap in the choosing, and it is the same one that spoils vendor benchmarks. If you sweep candidate thresholds and pick the one with the best observed precision, you have picked whichever threshold got lucky on a finite sample, and it will regress. Choosing on the lower bound of the interval rather than on the point estimate is what stops a threshold selected in evaluation from disappointing in production. The sweep also has to re-run the whole cascade at each candidate threshold rather than re-thresholding cached scores. Raising the upper threshold does not merely shrink the automatic band — it pushes those pairs into escalation, and only a real run shows what happens to them next. A sweep over cached probabilities silently assumes the escalated pairs behave like the automatic ones, which is precisely the assumption under test. ## Who does the work an abstention creates The objection to abstention is that it pushes work back onto the customer. That objection is right about the work and wrong about the alternative. The work does not disappear when a system decides everything; it moves from a queue you can see into wrong entities you cannot, and it arrives later and costs more. There is a second, better objection: that abstention can be used to hide a weak model. It can, and two numbers together settle it. A system that abstains on everything hard shows a high precision and a low automation rate, so publishing both makes the manoeuvre visible immediately, and neither figure may travel without the other. The design question that follows is which records should be abstained on, and the answer is the ones whose evidence genuinely does not discriminate — not the ones the model happens to find hard. Those are different populations. A truncated company name whose missing tokens were the discriminating ones cannot be resolved by a better model, because the information is not present in the input. You can watch that in the measurement. On the live-adjudicated 317-company fixture, the truncated-name shape resolves 58.10% (61/105). 44 of 105 come back as an abstention with a shortlist, which is the correct answer for a name whose missing tokens were the discriminating ones. ## Degrading to abstention rather than to error The test of an abstention design is what happens when the input is wrong rather than merely thin. A record whose free-text context names a town the company is not in is actively misleading, and the failure a naive system produces is a confident match to whichever company really is in that town. We measure this deliberately. In the historical 50-record wrong-town fixture, 72% resolved and zero wrong bindings were observed. This is a measured adversarial sample, not a guarantee for every misleading description. That is the property worth designing for: false context should cost coverage, never correctness. The mechanism is that caller-supplied prose ranks candidates rather than filtering them, so a wrong town demotes the true company without removing it from the list. The same principle runs through the acceptance rule. A match below 0.90 confidence is unrepresentable. The record comes back as an abstention with a ranked shortlist instead. It is an acceptance contract, not calibration evidence: it says what we refuse to return, not how often an accepted match is right. Read it with the coverage it costs. ## What an abstention has to carry An abstention that returns a null is a worse outcome than a guess, because it gives the caller nothing to act on and no way to improve the input. The contract has to be richer than that, and there are four things it should carry. - The ranked candidates the system considered, so a human or an agent can look at what it looked at. - A decision identifier that redeems the full evidence trail — features, thresholds, model and prompt versions — so the abstention is auditable rather than merely reported. - A route into review, with an identifier that persists, so the outcome of that review becomes a labelled example rather than a lost decision. - A statement of what would have changed it. "A postcode column would resolve this shape" is actionable; "unresolved" is not. ## Reading the shortlist correctly The ranked shortlist is the part of an abstention a reviewer actually works with, and its numbers mean something narrower than they look. Two figures describe it, and both are properties of the list rather than of any answer. Recall is a property of the list, not of the winner. 74% is far below the 0.90 acceptance floor, so no row of that shortlist may be offered as a match, quoted as an accuracy number, or folded into an accepted-match count. Concretely: on 205 name-only records on the gated acquisition fixture. the true company is at rank one for 74% (151/205) and inside the top three for 100% (205/205). Those figures come from the gated acquisition fixture — The stub-adjudicated run behind the pairwise gate, not the 317-company live recording behind the tier table. Read quickly, 74% (151/205) looks like an accuracy figure. It is the recall of a list. Offering the top row as a match is exactly the error the confidence floor exists to prevent, which is why the shortlist comes back as an abstention rather than as a low-confidence answer with a score attached. ## Measuring a system that declines Four numbers, together, make an abstaining system legible. Fewer than four and something has been hidden. - Automation rate: the share of records or pairs decided without human involvement. - Precision on decided outcomes: how often the automated decisions are right, with an interval and a sample size. - Coverage by input shape: which input shapes get answered and which abstain, because the blend is a property of the fixture rather than of the engine. - Shortlist recall: for the abstentions, how often the right answer was in the list — measured as a property of the list, never folded into an accuracy figure. ## What to take away - Entity resolution is a three-way trade. Once abstention exists, precision and coverage decouple, and precision becomes a statement about a chosen operating point. - False merges and false splits cost different amounts by roughly an order of magnitude, so the right operating point is rarely the one that maximises F1. - Abstention can hide a weak model, which is why precision and automation rate must always be published together — either alone is a much weaker claim. - Wrong or misleading input should degrade to abstention, not to error. Caller context ranks candidates rather than filtering them, for exactly this reason. - Shortlist recall is a property of the list. 74% (151/205) at rank one is not an accuracy figure, and no row of a shortlist may be offered as a match. Those takeaways hold for anybody's data. The number for yours comes out of your own rows — a few hundred of them, with the matches you already have beside them. [Start free →](https://app.entitybind.com/app?signup=1) ## Read next Each article names its own further reading. These pick up a question this one raises and leaves open. - [How to measure entity resolution accuracy](https://entitybind.com/guides/how-to-measure-entity-resolution-accuracy) (13 min read): Precision, recall, abstention, pairwise versus B-cubed cluster metrics, Wilson intervals and out-of-fold evaluation. Why a single accuracy number for entity resolution is always meaningless, with worked arithmetic. - [Fellegi-Sunter explained, with worked arithmetic](https://entitybind.com/guides/fellegi-sunter-explained) (10 min read): The probabilistic record linkage model behind most matching engines: m and u probabilities, match weights, the three decision regions, and a worked example you can check with a calculator. - [Calibration and confidence scores](https://entitybind.com/guides/calibration-and-confidence-scores) (10 min read): What a calibrated confidence actually means, how expected calibration error is computed, why a similarity score is not a probability, and why an LLM's self-reported confidence is neither. Or try your own rows: create your Free workspace and upload a company CSV to inspect its results. Contact us for an assisted measurement against your labelled data, with the working handed back. [Start free →](https://app.entitybind.com/app?signup=1) ## Where this article is applied 4 pages name this method as the working behind what they claim. - [CRM and RevOps](https://entitybind.com/use-cases/crm-and-revops) (Use case): Which accounts are one company, which merges to refuse, and a receipt for both. - [AI agents](https://entitybind.com/use-cases/ai-agents) (Use case): A counterparty check before the agent acts, and a receipt for what it acted on. - [Entity Bind vs Fuzzy matching](https://entitybind.com/compare/fuzzy-matching) (Comparison): Free, already there, and perfectly good until a wrong merge costs money. - [Entity Bind vs Screening tools](https://entitybind.com/compare/sanctions-screening-tools) (Comparison): We are not a screening tool and must not be used as one. This page is scope, not competition. ## Is this your file? The article is for anyone. The measurement behind it is worth commissioning for one kind of team. ### Built for Teams with a company master they have to defend — an MDM or stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, or counterparty onboarding that has to survive an audit. ### Wrong for Buying breadth as the product: we hold a fraction of an incumbent’s records and are not building a global company graph. Entity Bind resolves companies and organisations only. It does not resolve people, vessels, bank-account ownership, sanctions exposure, emissions or geolocation, and its results may not be used for eligibility decisions about individuals, including employment, credit and tenancy. ## Sources The papers, regulators and registers this article quotes, each with the caveat that travels with it. - [Fellegi and Sunter (1969), “A Theory for Record Linkage”, Journal of the American Statistical Association 64(328), 1183–1210](https://doi.org/10.1080/01621459.1969.10501049): The paper behind the three-region rule and the m and u probabilities. Its optimality result holds under the model's assumptions, conditional independence between fields among them, which real company data usually breaks. ## The basis for every figure above Internal fixtures only. The historical 2026-08-29 per-tier recording covers 317 GB companies with identifier-rich arrivals seeding the catalogue first; the context tier includes country and registered-town prose. Its legacy replay has no full-prompt-verified calls and cannot certify current calibration. The September 5 CI reports cover 1,176 labelled pairs and 705 records over 250 companies using offline stubs. These are not independent customer holdouts or live production validation; the 99% minimum-input product target remains unmet. Scope is company and organisation identity only — not people, vessels, bank-account ownership, sanctions or emissions. Results may not be used for eligibility decisions about individuals, including employment, credit and tenancy. Figures read off the product repository on 2026-09-05. The per-tier rates come from the live-adjudicated 317-company recording of 2026-08-29; the gate figures come from the stub-adjudicated continuous-integration run and are a floor rather than a ceiling. The two are never averaged. Entity Bind holds no SOC 2, no ISO 27001 and no other certification, and claims none. Every figure, with the population it was measured over and the method behind both recordings, is set out on [the accuracy page](https://entitybind.com/accuracy). The retention, erasure, tenancy and sub-processor detail a security review asks for is under [Trust and security](https://entitybind.com/trust), with where assurance stands: no external validation of the accuracy figures and no customer holdout. ## The same measurement, on your own rows. Create your workspace, verify your email and run your first company file. The Free plan needs no card. Create API keys in your workspace when you need an integration. [Start free](https://app.entitybind.com/app?signup=1) · [Plans and prices](https://entitybind.com/pricing) 300 credits. No card. For a guided review of your file, explore [the Merge Safety Sprint](https://entitybind.com/audit-pack). ## The questions this article answers ### Is abstention just a way of hiding a weak model? It can be, and the defence is publishing both numbers. A system that abstains on everything hard shows high precision and a low automation rate, and the two figures together make that visible immediately. Perfect precision at zero automation is trivial, which is why neither number is ever published here without the other. What abstention is legitimately for is the population where the evidence genuinely does not discriminate — a truncated name whose missing tokens were the ones that mattered cannot be resolved by a better model. ### Should I prefer precision or recall in entity resolution? Precision, in almost every commercial context, because the two errors cost different amounts. A false merge joins two companies into one entity, corrupts every aggregate computed over it, is discovered late, and is expensive to unwind. A false split leaves a duplicate that operations will notice and a later pass can fix. That asymmetry is why well-built resolvers prefer under-merging and why F1 is the wrong optimisation target here. ### What should an API return when it cannot resolve a record? An abstention with a ranked shortlist, a decision identifier that redeems the full evidence trail, a review route, and ideally a statement of what would have changed the answer. A match below 0.90 confidence is unrepresentable. The record comes back as an abstention with a ranked shortlist instead. A null teaches the caller nothing and a low-confidence guess is worse than both. ### What happens if the context I send is wrong? In the historical 50-record wrong-town fixture, 72% resolved and zero wrong bindings were observed. This is a measured adversarial sample, not a guarantee for every misleading description. ### Can I use the top shortlist candidate as a match? No, and the reason is in the number rather than in policy. Recall is a property of the list, not of the winner. 74% is far below the 0.90 acceptance floor, so no row of that shortlist may be offered as a match, quoted as an accuracy number, or folded into an accepted-match count. On 205 name-only records on the gated acquisition fixture. the true company is at rank one for 74% (151/205), which is well below the acceptance floor. The shortlist is there so a person or an agent can adjudicate with the same evidence the system had, not so the first row can be treated as an answer. --- Historical results on an internal UK fixture with a preloaded catalogue. The September 5 replay has no full-prompt-verified calls and cannot establish current calibration. No independent customer holdout or global coverage claim. This is the Markdown copy of https://entitybind.com/guides/precision-recall-and-abstention, assembled from the same data the page renders and carrying every sentence, table row, figure and question the page's main content carries. The HTML page is canonical; it adds the layout, the navigation and the interactive parts (the ask panel, the forms), which a copy can name but not run. Every page on the site, with a line on each: https://entitybind.com/llms.txt.