The integrity disciplines surveyed above take a value as given and certify its handling. This section turns to the field that produces the value in the first place, and to the signal that value arrives with. When a machine learning model emits an output, it emits a confidence alongside it, and the temptation is to treat that confidence as the missing basis: if the model is ninety percent sure, surely that is the warrant to act. The trustworthy machine learning literature, read in full and against itself, establishes the opposite. Confidence, in every form the field can produce it, answers a distributional question about the estimation process, not the question admissibility asks about a particular claim and a particular action. This is not a defect that better tooling will remove; the field's own most rigorous methods and its own survey say as much.
Where the confidence signal comes from, and why it misleads
The problem begins with the nature of the signal. A model's confidence, when it is baseless, is baseless by design rather than by malfunction. The analysis of why language models produce fluent falsehoods traces the behaviour to training objectives that reward confident guessing over calibrated abstention, so that basis-free assertion is a structural product of how the models are optimised, not an occasional error (Kalai et al., 2025). The philosophical companion to this point names the resulting output precisely: a system indifferent to the truth of what it emits produces assertions whose fluency is unconnected to their grounding (Hicks et al., 2024). And the signal that is supposed to let a receiver tell grounded from ungrounded output does not do so reliably: large models are systematically overconfident when their stated certainty is measured against their correctness (Groot and Valdenegro-Toro, 2024). The hallucination surveys establish that this is a first-class, studied phenomenon rather than a marginal one, with its own taxonomy of forms and causes (Huang et al., 2023) (Zhang et al., 2023). The confidence a value arrives with, then, is exactly the signal least able to certify the value's basis.
The apparatus, rung by rung
The field has built a substantial apparatus to represent what a model does not know, and it is worth walking it, because a supervisor is entitled to ask whether some rung of it supplies what raw confidence cannot. It does not, and each rung fails toward the same limit. The most-cited method for representing model uncertainty casts test-time dropout as approximate Bayesian inference, and its authors state the underlying problem in terms that are the thesis in the field's own words: softmax outputs are "often erroneously interpreted as model confidence", and a model "can be uncertain in its predictions even with a high softmax output" (Gal and Ghahramani, 2016). The scalable alternative, the deep ensemble, reads uncertainty from the disagreement of independently trained models, and carries the same caution the review leans on throughout: a prediction "may be accurate and yet miscalibrated, and vice versa", so accuracy and the quality of the uncertainty are orthogonal (Lakshminarayanan et al., 2017). A third route abandons the softmax entirely, placing a Dirichlet distribution over class probabilities to represent accumulated evidence and, in principle, to let a model say it does not know; even so, what it yields is an evidence-mass distribution over classes, a richer uncertainty representation and still not a per-action warrant (Sensoy et al., 2018). Figure 2 sets out the apparatus as the field built it, each rung reaching further than the last, and marks the line that none of them crosses.
Beneath these sits a prior question, whether the input is even the kind of thing the model was trained on, and the foundational baseline for it delivers the starkest statement of the whole problem: softmax probabilities "are not directly useful as confidence estimates", and random Gaussian noise fed to an image classifier draws a predicted class probability of ninety-one percent (Hendrycks and Gimpel, 2017). A model will assert a class, confidently, about pure noise, and nothing in the output distinguishes that case from a well-founded one. Two abstention mechanisms complete the apparatus by letting a model decline: a network with an integrated reject option that trades coverage for selective accuracy (Geifman and El-Yaniv, 2019), and the learning-to-defer formulation in which the model hands specific cases to a human expert (Mozannar and Sontag, 2020). These are the closest the field comes to admissibility, and the difference is instructive. Selective prediction does gate on the confidence signal whose insufficiency is the problem. Learning to defer need not: it can learn a comparative judgement of when the expert is likelier to be right than the model, rather than reading the model's own confidence. Yet both still decide only whether to act on the model's output, at the model, and neither carries a representation of the claim's basis across a boundary to a receiver who is not the model itself. They optimise the decision to defer; they do not make the basis inspectable by whoever receives the output.
The most rigorous rung is still marginal
The strongest witness is the field's most rigorous uncertainty tool, because it is genuinely rigorous and still does not answer the admissibility question. Conformal prediction wraps any pretrained model to produce prediction sets guaranteed to contain the true label with a user-specified probability, but the guarantee is explicitly marginal, averaged over the randomness in the calibration and test points, so a ninety percent coverage rate means roughly one prediction in ten is wrong and the method does not say which (Angelopoulos and Bates, 2021). The decomposition of uncertainty into its aleatoric and epistemic parts sharpens the diagnosis rather than closing the gap: confidence is not one thing but a conflation of irreducible noise in the world with the model's own reducible ignorance, and neither component is evidence that the asserted fact holds (Kendall and Gal, 2017). Even at its most rigorous, the apparatus makes statements about the long-run behaviour of an estimation procedure over a distribution, not about whether this claim's basis suffices for this act.
Calibration does not rescue confidence
The last line of defence is calibration, the promise that a well-calibrated model's ninety percent predictions are right ninety percent of the time. The review earlier relied on the finding that modern networks are badly miscalibrated (Guo et al., 2017), and honesty requires reporting that this finding has been complicated. The measure itself is contested: expected calibration error, the standard metric, "has numerous flaws", and which recalibration method looks best depends on arbitrary choices of binning and norm (Nixon et al., 2019). The headline result does not fully generalise: a systematic re-examination finds that the most recent architectures are among the best calibrated, and that the decay of calibration with model size and shift is less pronounced than the earlier work suggested (Minderer et al., 2021). This complication is conceded rather than hidden, and it costs the argument nothing, because aggregate calibration is a statement about a population of predictions and admissibility is a question about one. The decisive finding is that even the reassurance calibration does offer evaporates where it is most needed: calibrating on held-out data "does not guarantee calibration on shifted data", and under distributional shift most methods "give high confidence predictions on data that is entirely out-of-distribution", that is, they are confidently wrong about wholly unfamiliar inputs (Ovadia et al., 2019). Calibration holds when the input resembles the training data, which is when the stakes are lowest, and dissolves as the input becomes novel, which is when a receiver most needs to know whether to act.
The field's own verdict
The conclusion of this section is not the review's alone. The comprehensive survey of uncertainty in deep neural networks catalogues the whole apparatus, the Bayesian, ensemble, evidential, and single-deterministic families, and the calibration literature that tries to make their outputs trustworthy, and states the verdict plainly: estimating predictive uncertainty "is not sufficient for safe decision-making", and the field cannot reliably distinguish in-domain from out-of-domain inputs or provide dependable uncertainty for an individual decision (Gawlikowski et al., 2021). Every rung of the apparatus, from raw confidence through ensembles, evidential methods, out-of-distribution detection, conformal sets, the aleatoric and epistemic decomposition, and calibration, answers a distributional or procedural question about the estimation process. None carries, with a particular output, a judgement of whether its basis suffices for a particular irreversible action. That is a category difference, confirmed at the level of the field's own survey, and it correctly positions uncertainty quantification as a potential input to admissibility rather than a competitor to it: a conformal set or an epistemic-uncertainty estimate is exactly the kind of thing a basis representation could carry, but it does not by itself constitute the judgement.
References
Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. arxiv.org/abs/2509.04664
Hicks, M. T., Humphries, J. and Slater, J. (2024). ChatGPT is Bullshit. Ethics and Information Technology, 26(2). doi.org/10.1007/s10676-024-09775-5
Groot, T. and Valdenegro-Toro, M. (2024). Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models. arXiv:2405.02917. arxiv.org/abs/2405.02917
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B. and Liu, T. (2023). A Survey on Hallucination in Large Language Models. arXiv:2311.05232. arxiv.org/abs/2311.05232
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Xu, C., Chen, Y., Wang, L., Luu, A. T., Bi, W., Shi, F. and Shi, S. (2023). Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv:2309.01219. arxiv.org/abs/2309.01219
Gal, Y. and Ghahramani, Z. (2016). Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. ICML 2016. arXiv:1506.02142. arxiv.org/abs/1506.02142
Lakshminarayanan, B., Pritzel, A. and Blundell, C. (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. NeurIPS 2017. arXiv:1612.01474. arxiv.org/abs/1612.01474
Sensoy, M., Kaplan, L. and Kandemir, M. (2018). Evidential Deep Learning to Quantify Classification Uncertainty. NeurIPS 2018. arXiv:1806.01768. arxiv.org/abs/1806.01768
Hendrycks, D. and Gimpel, K. (2017). A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. ICLR 2017. arXiv:1610.02136. arxiv.org/abs/1610.02136
Geifman, Y. and El-Yaniv, R. (2019). SelectiveNet: A Deep Neural Network with an Integrated Reject Option. ICML 2019. arXiv:1901.09192. arxiv.org/abs/1901.09192
Mozannar, H. and Sontag, D. (2020). Consistent Estimators for Learning to Defer to an Expert. ICML 2020. arXiv:2006.01862. arxiv.org/abs/2006.01862
Angelopoulos, A. N. and Bates, S. (2021). A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. arXiv:2107.07511. arxiv.org/abs/2107.07511
Kendall, A. and Gal, Y. (2017). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? NeurIPS 2017. arXiv:1703.04977. arxiv.org/abs/1703.04977
Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML 2017. arXiv:1706.04599. arxiv.org/abs/1706.04599
Nixon, J., Dusenberry, M., Jerfel, G., Nguyen, T., Liu, J., Zhang, L. and Tran, D. (2019). Measuring Calibration in Deep Learning. CVPR Workshops 2019. arXiv:1904.01685. arxiv.org/abs/1904.01685
Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D. and Lucic, M. (2021). Revisiting the Calibration of Modern Neural Networks. NeurIPS 2021. arXiv:2106.07998. arxiv.org/abs/2106.07998
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B. and Snoek, J. (2019). Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. NeurIPS 2019. arXiv:1906.02530. arxiv.org/abs/1906.02530
Gawlikowski, J., Njieutcheu Tassi, C. R., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R. and Zhu, X. X. (2021). A Survey of Uncertainty in Deep Neural Networks. Artificial Intelligence Review. arXiv:2107.03342. arxiv.org/abs/2107.03342