Blocking

Blocking is the step that decides which pairs are worth comparing at all. Comparing every record against every other is quadratic: a million records is five hundred billion pairs, and no scorer is cheap enough for that. Blocking partitions or indexes the data so that only records sharing a blocking key are compared — the first four characters of the normalised name, the postcode, a phonetic code, a token, or several of these in parallel as separate passes.

Every blocking scheme trades recall against cost. A tight key produces few pairs and misses true matches whose key differs by one character; a loose key produces so many pairs that the saving disappears. Multiple independent passes are the standard answer, because a true pair only has to survive one of them, and a record with a mistyped postcode can still be caught by its name pass.

The crucial property is that blocking sets a ceiling on everything downstream. A true match that never becomes a candidate cannot be recovered by any scorer, however good — which is why blocking recall is measured separately from match accuracy, and why an accuracy figure quoted without a retrieval figure is quoting the second half of a pipeline.

Entity Bind weights every blocking key by how rare the token is across the catalogue and the live registers — 'services' is worthless as a key, 'Vodafone' is decisive — which is what rarity-weighted blocking means, and it measures retrieval recall as a metric in its own right. Treating those two tokens equally spends the candidate budget on the wrong records.