The companion entry surveyed machine learning's uncertainty apparatus and reported the field's own verdict that none of it is sufficient for safe decision-making. This entry narrows to the one part of that apparatus most likely to be offered as a rebuttal: calibration. If a model is well calibrated, the argument runs, then its ninety-percent predictions really are right ninety percent of the time, so the confidence number can be trusted after all. An earlier finding cited the calibration literature to establish that modern networks are miscalibrated. Intellectual honesty requires following that literature further, because the follow-up complicates the picture, and the complication turns out to strengthen the thesis rather than weaken it.
The reassurance, and the first crack: the measure itself
Calibration is the promise that predicted confidence matches empirical accuracy, and it is measured, almost universally, by expected calibration error. The first problem is that the yardstick is unsound. The comprehensive study of how calibration is measured finds that expected calibration error, the most popular metric, has numerous flaws, and that the rank ordering of which recalibration methods look best is drastically impacted by arbitrary choices such as the number of bins and the norm used (Nixon et al., 2019). This matters before any claim about whether a model is calibrated can even be stated: the instrument that certifies calibration gives different answers depending on how it is configured. A reassurance is only as good as its measurement, and this measurement is contested by the people who study it.
The second crack: the finding does not generalise
The review earlier leaned on the well-known result that modern neural networks are badly miscalibrated, larger and more accurate models being worse. Honesty requires reporting that this result has been revisited and does not hold across the board. The systematic re-examination of calibration in current architectures finds that the most recent models, notably those not built on convolutions, are among the best calibrated, and that the earlier trend of calibration decaying with model size and with distribution shift is less pronounced in recent architectures (Minderer et al., 2021). This is a genuine complication of a source this review relied on, and it must be held rather than hidden. But note what it does and does not say. It says some modern models are well calibrated in aggregate, on data drawn like the test set. It does not say that a well-calibrated model's individual high-confidence output carries a basis for the action taken on it. Aggregate calibration is a statement about a population of predictions; admissibility is a question about one. Better calibration narrows one gap the thesis never depended on and leaves the load-bearing gap exactly where it was.
The decisive crack: calibration dissolves under shift
The reassurance fails hardest precisely where it would need to hold. The large-scale benchmark of predictive uncertainty under dataset shift finds that calibrating on the validation set leads to well-calibrated predictions on the test set, but does not guarantee calibration on shifted data, and that traditional post-hoc calibration such as temperature scaling falls short as inputs move away from the training distribution (Ovadia et al., 2019). The finding that lands the blow is about confident error on genuinely novel inputs: most methods give high-confidence predictions on data that is entirely out of distribution, that is, they are confidently wrong about completely out-of-distribution data. The reassurance holds when the input resembles training data, which is exactly when the stakes are lowest, and it evaporates as the input becomes unfamiliar, which is exactly when a receiver most needs to know whether to act. Calibration is a fair-weather guarantee, and irreversible actions are not taken only in fair weather.
What this establishes
Calibration does not rescue confidence as a category. The metric that certifies it is contested; the headline miscalibration result does not generalise, which is worth conceding and costs the thesis nothing; and the guarantee that survives both caveats is marginal and fair-weather, dissolving under the distribution shift that accompanies high-stakes novelty. Even a perfectly calibrated model, on data it was calibrated for, tells a receiver only that a population of its ninety-percent claims is right about ninety percent of the time. It does not tell the receiver whether this claim, about this irreversible action, rests on a basis sufficient to proceed. That per-claim, action-relative judgement is what calibration was never built to provide and what this work exists to carry across the boundary.
References
Nixon, J., Dusenberry, M., Jerfel, G., Nguyen, T., Liu, J., Zhang, L. and Tran, D. (2019). Measuring Calibration in Deep Learning. CVPR Workshops 2019. arXiv:1904.01685. arxiv.org/abs/1904.01685
Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D. and Lucic, M. (2021). Revisiting the Calibration of Modern Neural Networks. NeurIPS 2021. arXiv:2106.07998. arxiv.org/abs/2106.07998
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B. and Snoek, J. (2019). Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. NeurIPS 2019. arXiv:1906.02530. arxiv.org/abs/1906.02530