Fellegi-Sunter explained, with worked arithmetic
For the team that owns a company master and has to defend it.
A stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, counterparty onboarding that has to survive an audit. Read the argument here, take the takeaways at the foot into your next vendor conversation, and send us a file when you want the same measurement on your own rows.
How matching works · 1,980 words · every figure carries the population it was measured over
10 min read, last revised . 5 questions answered at the foot.
all 19guides →how every figure here was measured →On this page
The problem Fellegi and Sunter were solving
Two files, no shared identifier, and a requirement to state how often the join is wrong, because the analysis downstream depends on it. That was the setting — official statistics — when Ivan Fellegi and Alan Sunter published "A Theory for Record Linkage" in the Journal of the American Statistical Association in 1969, formalising an approach Halbert Newcombe and colleagues had described in Science a decade earlier.
Their framing was decision-theoretic rather than algorithmic. Given a comparison vector between two records, choose one of three actions — link, do not link, or refer for clerical review — subject to bounds on the two error rates you care about. The result they proved is that a rule based on a single likelihood ratio, cut at two thresholds, minimises the size of the review region for any fixed pair of error bounds. That is why the three-region structure is not a design preference. It is optimal under the model's assumptions.
m and u probabilities
For each field you compare, two conditional probabilities define its value as evidence.
The m probability is the chance the field agrees given that the two records refer to the same entity. It is bounded below 1 by real-world messiness: a company moves, so its registered postcode changes; someone types a trading name in a legal-name field; a domain is recorded with a different subdomain. A field with a low m probability is one whose disagreement means very little.
The u probability is the chance the field agrees given that the two records refer to different entities. This is essentially the field's collision rate in the population, and it is where rarity lives. Two records agreeing on the legal form "Limited" have a u probability near one, because most of the register is Limited. Two records agreeing on an unusual name token have a u probability near zero.
The weight of an agreement is log₂(m/u); the weight of a disagreement is log₂((1−m)/(1−u)). Agreement on a field with high m and low u is worth a great deal. Agreement on a field with high u is worth almost nothing, however high its m probability is. That single asymmetry is the reason a rarity-blind similarity score systematically overrates common names.
Worked arithmetic
Take three comparable fields and give them illustrative m and u values. These numbers are chosen to make the arithmetic checkable and are not measurements of anything — real m and u values are estimated from data, and the section after this one covers how.
| Field | m | u | Agreement weight log₂(m/u) | Disagreement weight log₂((1−m)/(1−u)) |
|---|---|---|---|---|
| Company name (normalised) | 0.95 | 0.0001 | +13.21 | −4.32 |
| Registered postcode | 0.80 | 0.0002 | +11.97 | −2.32 |
| Legal form | 0.98 | 0.35 | +1.49 | −5.02 |
Illustrative values, not measurements. Note that legal form has a large disagreement weight and a small agreement weight: because most companies share a legal form, agreement is uninformative while genuine disagreement is not.
Adding the weights up, and where each pair lands
Suppose the upper threshold is +14 and the lower is 0. Those cut-offs would in practice be derived by sweeping the whole cascade and choosing on an interval's lower bound rather than a point estimate, but fix them here to see the mechanics.
Pair one: names agree, postcodes agree, legal forms differ. Total = 13.21 + 11.97 − 5.02 = +20.16. Above the upper threshold, so link. Note what happened: two strong agreements outweighed a legal-form conflict, which is right — a company can genuinely be recorded as a plc in one file and a Ltd in another when a group's records are stale.
Pair two: names agree, postcodes differ, legal forms agree. Total = 13.21 − 2.32 + 1.49 = +12.38. Between the thresholds, so this pair goes to review or to a further stage. It is exactly the case where you want a third option: the names agree strongly, the addresses do not, and either "the company moved" or "these are two companies with the same name" would explain it.
Pair three: names differ, postcodes differ, legal forms agree. Total = −4.32 − 2.32 + 1.49 = −5.15. Below the lower threshold, so do not link, and the agreement on legal form correctly did almost nothing to rescue it.
Converting a total to a probability is one more step: the log-odds are the total in natural-log units plus a prior term, and the logistic function maps that to a probability. The prior term is not decoration, and it is covered below.
The three regions
Two thresholds partition the score line into three regions, and each has a different contract.
Fellegi and Sunter's optimality result is about the third region: for fixed error bounds, this rule makes it as small as it can be.
- Above the upper threshold: automatic link. This is the region whose precision is the number worth publishing, because it is the region where nobody checks.
- Below the lower threshold: automatic non-link. Its quality is measured separately as negative predictive value, and it is routinely under-reported — a system that wrongly rejects is quietly fragmenting entities.
- Between them: escalate. Classically this meant clerical review. In a modern cascade it can mean a structured second opinion from a model, or an agent that goes and finds evidence neither record contained, with human review as the final stop.
Estimating m and u without labelled data
Where do m and u actually come from? The elegance of the model is undercut by that question, and there are three answers with different failure modes.
The u probabilities are the easy half: they are close to the frequency of agreement in random pairs, which you can estimate directly from the data by sampling pairs, since almost all random pairs are non-matches. This also gives you value-specific u probabilities, so "Smith Ltd" and an unusual name are not treated as equally informative agreements. That refinement matters more in company data than in most domains.
The m probabilities are harder, because working them out means knowing which pairs are true matches. The classical answer is to let the fit infer that as it goes, treating match status as a hidden variable and iterating until the estimates settle — expectation-maximisation over the comparison vectors. It works, and it inherits an assumption discussed below. The alternative is labelled data, which is more reliable and much more expensive.
The modern practical answer is to fit the weights directly with a regularised logistic regression over a labelled set, which subsumes the m/u parameterisation and lets you add features that are not clean field comparisons. That is what our own scorer does. It also introduces every hazard of a fitted model — in particular that reporting accuracy on the set you fitted on is in-sample and optimistic, which is why the fitting is cross-validated with folds grouped by entity family.
The conditional independence assumption, and where it breaks
Summing weights is only valid if the field comparisons are conditionally independent given match status. They frequently are not, and the effect is systematic overconfidence: correlated fields double-count the same evidence, pushing scores towards the extremes and making a nominally calibrated model badly calibrated at the top of its range.
Company data has textbook examples. A shared domain and a shared email host are not independent signals — in our own feature set, every firing of the email-domain conflict feature was a strict subset of the domain conflict feature's firings, so there was no independent variation for a fit to estimate at all. The correct response was to hold that feature at zero and say so, rather than assign it a plausible-looking weight that would apportion one disagreement twice.
A second, sharper version of the same problem: a triple of features consisting of "both sides have the field", "they agree" and "they conflict" is linearly dependent by construction, because if both sides carry the field then it either agrees or it does not. Only the contrasts against "field absent" are identified, and only those may be quoted. An individual coefficient inside such a triple is whatever the regulariser used as a tie-break, and reading meaning into it is reading noise.
The prior, and why a fitted intercept does not travel
A likelihood ratio is not a probability until it meets a prior. In the Fellegi-Sunter framing this is the proportion of compared pairs that are true matches; in a logistic regression it is the intercept.
That proportion is a property of your candidate generator, not of the world. A labelled gold set built to exercise hard cases might be more than half matches; a recall-oriented blocker in production might produce one match in a thousand candidate pairs. Deploying a scorer fitted on the first against the second means every score is inflated, and no amount of threshold tuning corrects it — it just moves where the errors land.
The fix is arithmetic: shift the intercept by logit(π_production) − logit(π_training) and re-derive the thresholds against the shifted model. Ignoring this is one of the more common ways a matcher that validated beautifully arrives in production performing visibly worse than it did in the notebook.
What a modern cascade adds
Fellegi-Sunter is still the right skeleton, and three things sit around it in a production system.
Deterministic short-circuits come first, because a matching registration number scoped to its register or a matching LEI is not evidence to be weighed — it is an answer, and treating it as a feature invites a pile of weak positive signals to outvote it. The mirror rule is more important: conflicting strong identifiers must be decisive against a match, and the magnitude that makes them decisive has to be re-checked whenever the surrounding weights are refitted, because a held constant is expressed in the units of the model around it and a refit moves those units.
Escalation replaces clerical review with something cheaper before it reaches a human: a structured verdict from a model, versioned by prompt and by model identity and cached for deterministic replay, and then an agent that gathers evidence neither record contained. That last stage attacks a different error class from everything before it — evidence availability rather than model quality — which is why it can help where threshold tuning cannot.
And abstention is made a first-class outcome rather than a queue nobody staffed. The record comes back unresolved with its shortlist, and the review decision that follows becomes a labelled example, which is how the m probabilities of the next fit get better.
What to take away
- The m probability is agreement given a true match; the u probability is agreement given a non-match. The agreement weight is log₂(m/u) and the disagreement weight is log₂((1−m)/(1−u)).
- Rarity is what makes the model work: agreement on a common value like a legal form is worth almost nothing, whatever its m probability.
- Two thresholds, not one. Fellegi and Sunter proved the three-region rule minimises the review region at fixed error rates, which is the formal case for abstention.
- The conditional independence assumption breaks on correlated fields and produces overconfidence at the top of the score range, which is where automatic merges live.
- A fitted intercept encodes the training set's match base rate and does not transfer to a different candidate generator without a prior shift.
Those takeaways hold for anybody's data. The number for yours comes out of your own rows — a few hundred of them, with the matches you already have beside them.
Start freePrefer to put it to a person? Talk to us instead →
Where this article is applied
3 pages name this method as the working behind what they claim.
Is this your file?
The article is for anyone. The measurement behind it is worth commissioning for one kind of team.
Teams with a company master they have to defend — an MDM or stewardship queue, a CRM after a migration or a merger, a supplier file before a payment run, or counterparty onboarding that has to survive an audit.
Buying breadth as the product: we hold a fraction of an incumbent’s records and are not building a global company graph. Entity Bind resolves companies and organisations only. It does not resolve people, vessels, bank-account ownership, sanctions exposure, emissions or geolocation, and its results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Sources
The papers, regulators and registers this article quotes, each with the caveat that travels with it.
- Fellegi and Sunter (1969), “A Theory for Record Linkage”, Journal of the American Statistical Association 64(328), 1183–1210
The paper behind the three-region rule and the m and u probabilities. Its optimality result holds under the model's assumptions, conditional independence between fields among them, which real company data usually breaks.
- Newcombe, Kennedy, Axford and James (1959), “Automatic Linkage of Vital Records”, Science 130(3381), 954–959
The probabilistic approach before it was formalised, worked out on birth and marriage records. It describes linking records about people, which is outside this product's scope; it is cited for the method.
Internal fixtures only. The historical 2026-08-29 per-tier recording covers 317 GB companies with identifier-rich arrivals seeding the catalogue first; the context tier includes country and registered-town prose. Its legacy replay has no full-prompt-verified calls and cannot certify current calibration. The September 5 CI reports cover 1,176 labelled pairs and 705 records over 250 companies using offline stubs. These are not independent customer holdouts or live production validation; the 99% minimum-input product target remains unmet. Scope is company and organisation identity only — not people, vessels, bank-account ownership, sanctions or emissions. Results may not be used for eligibility decisions about individuals, including employment, credit and tenancy.
Figures read off the product repository on 2026-09-05. The per-tier rates come from the live-adjudicated 317-company recording of 2026-08-29; the gate figures come from the stub-adjudicated continuous-integration run and are a floor rather than a ceiling. The two are never averaged. Entity Bind holds no SOC 2, no ISO 27001 and no other certification, and claims none. Every figure, with the population it was measured over and the method behind both recordings, is set out on the accuracy page. The retention, erasure, tenancy and sub-processor detail a security review asks for is under Trust and security, with where assurance stands: no external validation of the accuracy figures and no customer holdout.
The same measurement, on your own rows.
Create your workspace, verify your email and run your first company file. The Free plan needs no card. Create API keys in your workspace when you need an integration.
300 credits. No card.
For a guided review of your file, explore the Merge Safety Sprint →
The questions this article answers
What are m and u probabilities in record linkage?
The m probability of a field is the chance it agrees given that two records refer to the same entity — bounded below 1 by moves, typos and stale data. The u probability is the chance it agrees given that they refer to different entities, which is essentially the field's collision rate in the population. The agreement weight is log₂(m/u) and the disagreement weight is log₂((1−m)/(1−u)), so a field that is common in the population contributes very little when it agrees no matter how reliable it is.
Why does Fellegi-Sunter use two thresholds instead of one?
Because the third outcome is worth having. Above the upper threshold you link automatically, below the lower you reject automatically, and between them you escalate rather than guess. Fellegi and Sunter's 1969 result is that for any fixed pair of error rates, this rule makes that middle region as small as it can be. It is the formal argument for abstention: given a precision requirement, sending the smallest possible slice for review is optimal rather than merely prudent.
Is Fellegi-Sunter still used, or has machine learning replaced it?
The framework is still the skeleton; the estimator has changed. Most production matchers now fit weights with regularised logistic regression or a gradient-boosted model over a labelled set, which subsumes the m/u parameterisation and allows features that are not clean field comparisons. What survives unchanged is the structure: rarity-weighted evidence summed into log-odds, two thresholds, and a third region that escalates.
What is the conditional independence assumption and why does it matter?
The model sums per-field weights, which is only valid if the field comparisons are independent given match status. Real fields correlate — a shared domain and a shared email host carry much of the same information — so summing them double-counts evidence and pushes scores towards the extremes. The practical effect is overconfidence exactly at the top of the range, where automatic merges are made. The safe response is to hold a correlated feature at zero rather than fit it a plausible-looking weight.
Can I take a scorer trained on one dataset and use it on another?
Not without adjusting the prior. The fitted intercept encodes the proportion of compared pairs that were true matches in the training population, and that proportion is a property of the candidate generator rather than of the world. Moving to a blocker with a different match rate requires shifting the intercept by logit(new base rate) − logit(old base rate) and re-deriving the thresholds. Skipping this is a common reason a matcher that validated well performs worse in production.