How to measure entity resolution accuracy
For the team that owns a company master and has to defend it.
A stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, counterparty onboarding that has to survive an audit. Read the argument here, take the takeaways at the foot into your next vendor conversation, and send us a file when you want the same measurement on your own rows.
Measurement · 2,810 words · every figure carries the population it was measured over
13 min read, last revised . 6 questions answered at the foot.
all 19guides →how every figure here was measured →On this page
Why accuracy alone is meaningless here
Accuracy is the proportion of decisions that were correct. In entity resolution the population of possible decisions is every pair of records, which is quadratic in the size of the file, and almost all of those pairs are non-matches. A file of ten thousand records contains just under fifty million pairs, of which perhaps a few thousand are true matches.
A system that answers "not a match" to every pair therefore scores above 99.99% accuracy while resolving nothing. That is not a contrived counterexample; it is the arithmetic of class imbalance, and it is why the metric is abandoned in this field rather than merely deprecated. Any published "99% accurate" claim for a resolver is either measuring something other than accuracy or measuring accuracy over a population that has been quietly rebalanced.
The same argument disqualifies the mirror-image number. "Match rate" — the share of submitted records that got an answer — says nothing about whether the answers were right, and is trivially maximised by lowering the bar. The two numbers are only meaningful in each other's company, which is the first of the five prohibitions this article is organised around.
Precision and recall, in resolution terms
Precision is the share of proposed matches that are correct. Recall is the share of true matches that were found. In a resolver they answer two commercially different questions, and confusing them is how a procurement goes wrong.
Precision failures are wrong merges. Two different companies are welded into one entity, and everything computed over that entity — exposure, spend, credit limit, duplicate count — is now wrong in a way that is expensive to find and expensive to unwind. In a compliance context a wrong merge converts into damages. Precision is therefore the number to hold a hard line on.
Recall failures are missed merges. The same company appears twice, the duplicate persists, and somebody eventually notices. This is a cost, not a catastrophe, and it is recoverable by a later pass with better evidence. The asymmetry is the reason well-built resolvers prefer under-merging, and it is the reason the interesting figure is precision on automated merges rather than F1.
F1 has a specific problem here beyond the usual one. It is the harmonic mean of two quantities whose costs differ by orders of magnitude in this domain, so optimising it trades away exactly the property you were buying. Report both numbers separately, and report the operating point you chose between them.
A match rate without an abstention rate is unreadable
A resolver has three outcomes, not two: it decided and was right, it decided and was wrong, or it declined to decide. Any evaluation reporting two of those three has hidden a dial that can be turned freely.
Perfect precision at zero automation is trivial — decline everything. So precision is never published here without the automation rate beside it. On the continuous-integration gate of 2026-09-05, automated merge precision is 1.0000 on 357/357 merges and the automation rate is 63.10%. 742 of 1,176 pairs were decided automatically in the September 5 saved gate report. This measures automation, separately from accepted-merge precision.
The same discipline applies to end-to-end resolution. That gate resolves 70.78% of records (499/705); the live-adjudicated 317-company recording of 2026-08-29 blends to 95.75%. Those two figures come from different instruments over different fixtures and are never averaged. End-to-end resolution on the internal gated acquisition fixture with offline stub models. All 499 accepted matches were correct in this fixture. The current 99% product-target diagnostic fails; this run does not measure live model conversion or independently establish calibration.
And a blend is itself a warning. A historical 2026-08-29 blend across six input shapes in a preloaded UK catalogue. Its value depends on that mix: truncated names measured 58.10% while the blend measured 95.75%. Neither establishes the current minimum-input target. The per-input-shape table is the measurement; the blend only summarises what a particular fixture was made of.
| Input shape | What the caller sends | Resolved | Fraction |
|---|---|---|---|
| Identifier | A company number or an LEI | 100% | 317/317 |
| Name and website | A company name and its domain | 100% | 197/197 |
| Name, country and town prose | A company name, GB, and a sentence naming the registered town | 99.37% | 315/317 |
| Name and postcode | A company name and a registered postcode | 98.74% | 313/317 |
| Name and country | A company name and nothing else but the country | 98.42% | 312/317 |
| Name and town | A company name, country and a typed registered town | 98.42% | 312/317 |
| Truncated name | A name whose ending has been cut off by a field-length limit | 58.10% | 61/105 |
Historical results on an internal UK fixture with a preloaded catalogue. The September 5 replay has no full-prompt-verified calls and cannot establish current calibration. No independent customer holdout or global coverage claim.
Pairwise metrics flatter. Cluster metrics do not.
Pairwise precision counts decisions. Cluster metrics count records. The difference is not presentational, because one wrong pairwise decision does not cost one record — transitive closure spreads it across everything on both sides of the edge.
Work the arithmetic. Suppose the truth is two companies of three records each: {A, B, C} and {D, E, F}. The system correctly links A-B, A-C, B-C, D-E, D-F and E-F, and makes exactly one mistake, linking C to D. That is six correct decisions and one wrong one, so pairwise precision over the decisions it made reads 6/7, about 0.857. A single error in seven looks survivable.
But the clusters it produced are one cluster of six records. Every record in that cluster is now sharing an identity with three records it has nothing to do with. Charge the mistake to each record instead — that is the B-cubed convention — and for every one of the six, the predicted cluster has six members of which three are correct, so B-cubed precision is 3/6 = 0.5 for all six records, and 0.5 overall. B-cubed recall is 1.0, because every true partner was found. F1 is 0.667. One wrong edge took the record-level precision from 1.0 to 0.5, and the pairwise number never showed it.
The definition is plainer than the name. For each record, look at the cluster the system put it in and the cluster the truth puts it in. Its B-cubed precision is the share of its predicted cluster that genuinely belongs with it; its B-cubed recall is the share of its true cluster that ended up with it. Average each over all records. That per-record averaging is exactly why a large wrong merge is penalised in proportion to its size.
Cluster metrics also punish abstention
The same arithmetic runs the other way, and it explains an otherwise confusing figure. Take a true cluster of three records where the system resolves two of them and abstains on the third. An unresolved record is a predicted singleton, so the prediction is {A, B} and {C}.
B-cubed precision is 1.0 — nothing wrong was merged. B-cubed recall is not: A finds two of its three true partners, B finds two of three, and C finds only itself, one of three. The mean is (2/3 + 2/3 + 1/3) / 3 ≈ 0.5556, so F1 is about 0.714. A perfectly safe run scores badly on cluster F1 because safety and completeness are different properties.
That is what our own two cluster figures show, and why they are never averaged. On the live-adjudicated fixture, B-cubed cluster precision is 1.0000. On the stub-adjudicated gate, cluster F1 is 0.7584 — the arithmetic above rather than a worse engine, because every name-only record on that fixture abstains by construction and splits the family it belongs to. Both numbers are published, and a run described by its pairwise precision alone is a run you have only seen half of.
A rate without an interval is not a measurement
Every rate here is reported three ways: the observed value, the range the evidence actually supports at 95% confidence — computed the Wilson way — and the sample size behind both. The interesting rates in this field sit at or near 1.0, and that is exactly where the textbook normal-approximation interval fails: at p̂ = 1 it collapses to zero width and would "prove" perfection from thirty-nine observations. The Wilson interval stays finite and asymmetric there, which is what you want from an interval whose job is to stop you over-claiming.
At zero observed errors the Wilson lower bound has a closed form that is worth memorising, because it converts a claim into a required sample size in one line. With z = 1.96, the lower bound of n clean trials out of n is n / (n + z²) = n / (n + 3.8415).
Run it forwards and the consequence is unavoidable. To prove a 0.99 lower bound you need n / (n + 3.8415) ≥ 0.99, which means n ≥ 381. Not 100 clean merges, not 200 — 381 consecutive error-free automated merges, and one error resets the arithmetic to something harder. Our gate has 357/357 clean merges, which gives a lower bound of 98.94%. The 95% Wilson lower bound of 357 error-free automated merges. Proving a 99% lower bound needs at least 381 consecutive clean merges, so this is 24 short of it — and we publish the bound rather than the point estimate.
The same formula produces the calibrated confidence the API returns. The nominal 95% Wilson lower bound of 1,190 historical prompt-v8 uncorroborated MATCH verdicts with zero recorded errors. The 2026-08-29 population pools six input shapes with repeated company families and calibration/evaluation overlap. Its independence and transfer to new customer inputs are unproven. Legacy replay cannot certify current calibration. A published point estimate of 1.0000 with no denominator is not a stronger claim than a lower bound of 98.94%; it is the same evidence with the uncertainty deleted.
There is a second reason to publish the bound rather than the point estimate, and it is about which threshold you pick. If you sweep operating points and choose the one with the best point estimate, you are choosing whichever threshold got lucky on a small sample. Choosing on the interval's lower bound is how a calibrated 99% stops becoming 96% in production.
In-sample versus out-of-fold, the load-bearing caveat
Ask a supplier which labelled set the weights were fitted on. If it is the same set the accuracy is reported over, the headline is in-sample and it is optimistic. This is the second question worth asking, and the one that is much cheaper to ask now than to discover in diligence.
The leakage-free analogue is k-fold cross-validation, with one refinement that matters more in this domain than in most. Folds must be split by entity family, not by row. If two variants of the same company sit on opposite sides of a train/test boundary, the model has seen the answer, and the fold reports generalisation it has not demonstrated. Grouping by a stable family key — so an entity and its confusable siblings are held out together — is the only version of this that means anything.
The out-of-fold number is normally worse, and reporting the gap is what makes the in-sample number safe to publish. Our own pooled in-sample instrument proves the 98.94% lower bound quoted above; the family-grouped out-of-fold frontier over the probabilistic band alone proves a weaker one still. Neither proves 0.99, and saying so is more useful than a headline that quietly rests on whichever instrument flattered it.
The related trap is fitting the intercept. A model's intercept encodes the base rate of matches in the population it was fitted on, and a labelled gold set is usually far more balanced than a production candidate generator's output. Deploying without a prior shift means the scorer is systematically overconfident on real traffic, and no amount of threshold tuning fixes a wrong prior — it just moves where the errors land.
Stratify, or the failing stratum hides
A blended headline conceals the stratum that fails, and the stratum that fails is what diligence looks for. Four cuts are worth demanding from any evaluation, and their absence is itself informative.
- By evidence tier — the strongest class of evidence present on both sides of the comparison. Identifier-tier pairs are decidable at near-1.0 and name-only pairs are not, so a blend across tiers is a statement about the mix.
- By label provenance — handwritten adversarial cases against synthetically generated ones. Generated pairs are easier by construction: their perturbations come from a known operator set the normaliser was built for, and their negatives are always separable from the fields on the page. They exist to give the statistics a denominator large enough to support a claim at all, so ask for the handwritten row beside any figure quoted from the blended set.
- By hardness — easy, medium, hard, ambiguous — because a fixture's difficulty distribution is the single easiest thing for a vendor to choose favourably.
- By deciding stage — which part of the cascade spent the error budget. A threshold can only move the probabilistic band, so if the errors are in the deterministic stage, calibration work is tuning a dial that cannot reach them.
Denominators, and the two that get quietly swapped
Two denominators are routinely misread, and both misreadings favour the vendor.
The first is recall at k, the share of records whose true entity appears somewhere in the top k candidates. Its denominator is normally records whose true entity exists in the catalogue at all — a target that is not there cannot be retrieved, so the figure says nothing about first-arrival records. It is also a property of the list rather than of any answer. Our own name-only shortlist puts the right company at rank one for 74% (151/205) and inside the top three for 100% (205/205), on 205 name-only records on the gated acquisition fixture. Recall is a property of the list, not of the winner. 74% is far below the 0.90 acceptance floor, so no row of that shortlist may be offered as a match, quoted as an accuracy number, or folded into an accepted-match count.
The second is accuracy on decided outcomes versus accuracy over submitted records. On the live-adjudicated fixture: Zero wrong bindings were observed across the historical six-shape recording and the separate domain fixture. Observed correctness was 1.0000 on accepted outcomes; this does not guarantee an error-free new workload. That is a real and important result, and it is measured over the records the system chose to answer — which is exactly why it is published next to the per-tier resolution rate rather than instead of it.
What to demand from a vendor
Take these into the next supplier call. The five prohibitions our own accuracy document is written around double as a procurement checklist, because each one names a way a true sentence can be made misleading.
- Never a precision figure without the automation rate it was drawn from.
- Never a pairwise figure without the resolution-level figure beside it.
- Never a pairwise figure without cluster metrics beside it.
- Never a blended rate without the per-input-shape table it was blended from.
- Never a rate without its interval and its sample size.
- And one addition for buyers specifically: ask whether the fixture is internal. Ours are. Every figure on this site was measured on repository-authored fixtures, not a customer holdout and not independent third-party validation, and a supplier who will not tell you which they have is answering the question.
What to take away
- Accuracy over pairs is meaningless under class imbalance — answering "no" to everything scores above 99.99% — and match rate without abstention rate is equally unreadable.
- Precision failures are wrong merges and are expensive; recall failures are missed merges and are recoverable. Report them separately rather than as F1.
- One wrong pairwise edge can halve record-level precision through transitive closure: 6/7 pairwise becomes 0.5 B-cubed in the worked example above.
- Cluster F1 also punishes abstention, because an unresolved record is a predicted singleton. A safe run can score badly, and that is the metric working.
- At zero errors the Wilson lower bound is n/(n+3.8415), so a 0.99 lower bound requires 381 consecutive clean merges. 357/357 gives 98.94%.
- In-sample precision from weights fitted on the same gold set is optimistic. The leakage-free analogue is family-grouped k-fold, and it is normally worse.
Those takeaways hold for anybody's data. The number for yours comes out of your own rows — a few hundred of them, with the matches you already have beside them.
Start freePrefer to put it to a person? Talk to us instead →
Where this article is applied
4 pages name this method as the working behind what they claim.
Is this your file?
The article is for anyone. The measurement behind it is worth commissioning for one kind of team.
Teams with a company master they have to defend — an MDM or stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, or counterparty onboarding that has to survive an audit.
Buying breadth as the product: we hold a fraction of an incumbent’s records and are not building a global company graph. Entity Bind resolves companies and organisations only. It does not resolve people, vessels, bank-account ownership, sanctions exposure, emissions or geolocation, and its results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Sources
The papers, regulators and registers this article quotes, each with the caveat that travels with it.
- Bagga and Baldwin (1998), “Entity-Based Cross-Document Coreferencing Using the Vector Space Model”, COLING-ACL 1998
Section 7.3 sets out the B-CUBED scoring algorithm, written for cross-document coreference of people's names rather than for company records. The per-record averaging is the part this site borrows.
- Wilson (1927), “Probable Inference, the Law of Succession, and Statistical Inference”, Journal of the American Statistical Association 22(158), 209–212
The origin of the score interval. The zero-error closed form quoted on this site, n / (n + z²), is that interval's lower bound at an observed proportion of 1; the arithmetic is ours.
Internal fixtures only. The historical 2026-08-29 per-tier recording covers 317 GB companies with identifier-rich arrivals seeding the catalogue first; the context tier includes country and registered-town prose. Its legacy replay has no full-prompt-verified calls and cannot certify current calibration. The September 5 CI reports cover 1,176 labelled pairs and 705 records over 250 companies using offline stubs. These are not independent customer holdouts or live production validation; the 99% minimum-input product target remains unmet. Scope is company and organisation identity only — not people, vessels, bank-account ownership, sanctions or emissions. Results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Figures read off the product repository on 2026-09-05. The per-tier rates come from the live-adjudicated 317-company recording of 2026-08-29; the gate figures come from the stub-adjudicated continuous-integration run and are a floor rather than a ceiling. The two are never averaged. Entity Bind holds no SOC 2, no ISO 27001 and no other certification, and claims none. Every figure, with the population it was measured over and the method behind both recordings, is set out on the accuracy page. The retention, erasure, tenancy and sub-processor detail a security review asks for is under Trust and security, with where assurance stands: no external validation of the accuracy figures and no customer holdout.
The same measurement, on your own rows.
Create your workspace, verify your email and run your first company file. The Free plan needs no card. Create API keys in your workspace when you need an integration.
300 credits. No card.
For a guided review of your file, explore the Merge Safety Sprint →
The questions this article answers
Why is a single accuracy number for entity resolution meaningless?
Because the population of possible decisions is every pair of records and almost all of them are non-matches. A system that says "not a match" to everything scores above 99.99% accuracy on a ten-thousand-record file while resolving nothing. Accuracy is not a slightly weak metric here, it is a disqualified one. What you want instead is precision on automated decisions, the automation rate it was drawn from, cluster metrics, an interval, and the input shapes it was measured over.
What are B-cubed metrics and why do they matter?
B-cubed evaluates clusters record by record. For each record, its precision is the share of the cluster the system put it in that genuinely belongs with it, and its recall is the share of its true cluster that ended up with it; both are then averaged over all records. It matters because pairwise metrics undercount the cost of a wrong merge. In the worked example in this guide, a system making one wrong pairwise decision out of seven has pairwise precision 0.857 and B-cubed precision 0.5, because transitive closure spread that one edge across six records.
How many clean matches do you need to prove 99% precision?
381. At zero observed errors the 95% Wilson lower bound of n successes in n trials is n / (n + 1.96²) = n / (n + 3.8415), and that expression only reaches 0.99 at n = 381. Our own continuous-integration gate has 357/357 error-free automated merges, giving a lower bound of 98.94%, which is 24 short of the sample size the stronger claim would need. We publish the bound rather than the point estimate.
What is the difference between in-sample and out-of-fold accuracy?
In-sample means the model's weights were fitted on the same labelled set the accuracy is reported over, which makes the headline optimistic. Out-of-fold means the weights were fitted on one part of the data and evaluated on a held-out part — and in this domain the folds must be split by entity family, so a company and its confusable siblings are held out together. Splitting by row lets the model see the answer. Our out-of-fold frontier proves a weaker bound than our in-sample one, which is the normal and expected direction.
Why does your cluster F1 look so much worse than your pairwise precision?
B-cubed cluster F1 on the gate's own 705-record fixture, held as a regression ratchet against a 0.75 floor rather than offered as a headline. It reads far below the live recording's 0.9646 because an unresolved record is a predicted singleton and all 205 name-only arrivals on this fixture abstain by construction, splitting the families they belong to. The two cluster figures come from different instruments over different fixtures and are never averaged.
Are your accuracy figures independently validated?
No. Every fixture is one we built, in our own repository. There is no customer holdout and no third-party validation. What is checkable instead is the method: the fixtures, the adjudicator verdict logs and the harnesses are versioned, every figure is reproducible from a named script, and the failing numbers are published beside the passing ones.