Skip to content
Paula Livingstone writing · projects · tools

Attestable Review: Working Notes

Literature Review: What AI Governance Mandates, and Where It Stops

The AI governance frameworks (NIST AI RMF, the EU AI Act, the OECD recommendation, algorithmic auditing) all locate transparency and accountability at the handover, requiring that basis reach the receiver, and the NIST generative profile even names confabulation and over-reliance as official risks. But they mandate that the basis arrive; they do not adjudicate whether a particular claim suffices for a particular action.

AI governance is the field most likely to be mistaken for a rival to this work, because it uses the same words: transparency, accountability, explanation. It is worth being exact about what those instruments actually require, because when they are read closely they do not adjudicate a claim's basis at a handover. They mandate, repeatedly and with force, that the basis reach the receiver. That is a different thing, and the difference is the whole contribution.

The frameworks locate transparency at the handover

The governance instruments converge on a single structural move: the party that produces an automated output owes information to the party that will act on it. The United States framework organises trustworthiness around four functions, govern, map, measure and manage, and names being valid and reliable as the foundational characteristic on which the others rest, with accountability and transparency listed as distinct characteristics alongside it (NIST, 2023). The European regulation is more specific about the boundary. Its transparency provision requires that a high-risk system be sufficiently transparent to enable the deployer to interpret the system's output and use it appropriately (European Union, 2024). The transparency is owed to the deployer, and its stated purpose is that the deployer can judge the use. The international recommendation makes the same move from the side of the affected person: actors should provide meaningful information to enable those affected by an automated decision to understand the output and, crucially, to challenge it (OECD, 2019).

Read together, these are all statements about the handover. Each says the basis for an output must travel to whoever receives it, so that the receiver can interpret, use appropriately, or contest. That is exactly the boundary this work guards. But naming the boundary is not the same as building a gate at it. None of these instruments specifies a per-output, action-relative representation of whether a particular claim's basis suffices for a particular irreversible action, nor a mechanism by which a receiver, especially a machine receiver with no human present, could refuse an output on that ground. They mandate that the basis arrive; they do not adjudicate on it.

Accountability is process, not per-claim

The accountability strand of the field sharpens the same point. The most developed proposal for closing what its authors call the accountability gap is an end-to-end framework for internal algorithmic auditing, run across the development lifecycle before a model is deployed (Raji et al., 2020). Auditing of this kind is genuine and valuable, but it is process accountability: it establishes that the development process was carried out to a standard, documented, and reviewable. It does not, and is not built to, establish that a particular output's basis is sufficient for the particular act about to be taken on it. This is the same line the review has drawn in every neighbouring field, integrity of a declared process against admissibility of a claim's basis, now in the governance register. A model can pass its audit and still emit, at run time, a confident claim whose basis would not warrant the action a receiver is about to take on it.

The framework that names the failure

The strongest witness in this block is an official one, and it is striking because it names the thesis's own problem in its own risk taxonomy. The generative-AI profile of the United States framework enumerates, as official categories of risk, confabulation, defined as the production of confidently stated but erroneous or false content by which users may be misled or deceived, and human-AI configuration, under which it lists automation bias and over-reliance (NIST, 2024). It further treats content provenance and information integrity as core considerations for generative systems. The thesis's diagnosis, a confident baseless claim, its mechanism of harm, over-reliance at the point of handover, and the direction of its remedy, provenance of basis, all appear as named risks in an official framework. What the framework does not supply is the instrument. It catalogues confabulation and over-reliance as things to be managed; it does not carry a per-claim basis across the machine boundary so the over-reliance can be refused by something other than a human's judgement.

Where this leaves the positioning

AI governance mandates that basis reach the receiver, and its most recent official text even names the failure that follows when it does not. This is not a field that has missed the problem. It is a field whose instruments stop at the boundary this work is built to sit on. Its transparency is model-level and decision-level, owed so that a human can interpret, use, or contest; its accountability is lifecycle process; its risk taxonomy names confabulation and over-reliance without supplying a mechanism that adjudicates a claim's basis at the moment of action. The gap this work names is not outside the governance frameworks, waiting to be noticed. It sits inside their own stated aims, unmet by their own instruments. The companion entry turns to the other half of the field, the explanation methods themselves, and shows that even when an explanation is produced it does not recover the basis either.

References

NIST (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. doi.org/10.6028/NIST.AI.100-1

European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 13. Official Journal of the European Union L 2024/1689. eur-lex.europa.eu/eli/reg/2024/1689/oj

Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D. and Barnes, P. (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. FAT* 2020. arXiv:2001.00973. arxiv.org/abs/2001.00973

OECD (2019). Recommendation of the Council on Artificial Intelligence, Principle 1.3 (Transparency and explainability). OECD/LEGAL/0449. legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449

NIST (2024). Artificial Intelligence Risk Management Framework: Generative AI Profile. NIST AI 600-1. doi.org/10.6028/NIST.AI.600-1