Calibration

Accuracy asks whether the answers are right. Calibration asks whether the number attached to each answer means what it says. A model can be highly accurate and badly calibrated, in which case its confidence field cannot be used for routing, thresholds or automation decisions — which is usually the only reason it was exposed in the first place.

Calibration is measured by bucketing decisions by their stated confidence and comparing each bucket's stated rate to its observed rate, on held-out data. Raw model log-probabilities are almost never calibrated out of the box; the standard remedies are Platt scaling, isotonic regression, or, where a system records outcomes over time, deriving the confidence directly from observed trial and error counts.

The last approach is the one Entity Bind takes, and it has a property worth more than the value: the confidence values are pure functions of recorded trial and error counts with no tunable term, and the tests recompute every shipped value from its counts, so an edited constant fails the build. The confidence cannot quietly become a dial.

The headline calibrated confidence is 0.9968. The nominal 95% Wilson lower bound of 1,190 historical prompt-v8 uncorroborated MATCH verdicts with zero recorded errors. The 2026-08-29 population pools six input shapes with repeated company families and calibration/evaluation overlap. Its independence and transfer to new customer inputs are unproven. Legacy replay cannot certify current calibration. Confidence bounds depend on the recorded evidence population; there is no universal score for every match. Read match_confidence_basis and the evidence. A 0.99 filter can return some v8 adjudicated matches while excluding valid identifier matches, so validate its coverage cost on your own inputs. The reject direction is measured separately. The historical prompt-v8 rejection population has 54 verdicts and four errors. Its bound authorises no accepted match; other non-match populations carry their own values.