Skip to content
Paula Livingstone writing · projects · tools

Projects

Attestable

When probabilistic AI feeds safety-critical systems, unproven output can drive physical action unchecked. Attestable is a discipline for marking a claim's basis, so it cannot.

OT SecurityCyber-Physical AssuranceIEC 62443ProvenanceSafety CaseLLM Agents

Review: Working Notes

21 entries

Field-by-field working notes produced while building the review, one per area surveyed. These are the raw findings behind the polished Literature Review chapter, kept for transparency, not the chapter itself.

  1. Literature Review: Provenance (W3C PROV) The first of the near-neighbour readings. PROV records where information came from but adjudicates nothing; its own definition names trust assessment as provenance's purpose and then places that assessment outside itself, marking the exact edge of the gap.
  2. Literature Review: Policy Engines (OPA, Rego) The cleanest of the near-neighbour readings. A policy engine evaluates whatever rules and facts it is given and supplies neither; to make one refuse False Determinism you must first build the basis model, which is the contribution. OPA is where Attestable would run, not a rival to it.
  3. Literature Review: Attestation (in-toto, SLSA) The last near-neighbour reading, and the sharpest. Attestation frameworks certify who produced an artefact and that it was unaltered; they say nothing about whether it is true or fit to act on. Integrity is orthogonal to admissibility, and both specs state it against themselves.
  4. Literature Review: The Gap, Discharged Where the three readings come together. Provenance, attestation, and policy each stop exactly short of representing a claim's basis and adjudicating fitness for an action; even composed, they leave that model un-supplied. The gap the problem statement asserted is now the reading's conclusion.
  5. Literature Review: Why the Failure Is Structural The other half of the review. Four findings assemble to show False Determinism is not a passing quirk: generation produces basis-free assertion by design, confidence is the wrong category to fix it, the deficit is an absence rather than a concealment, and handover strips what little remains.
  6. Literature Review: Confidence, Calibration, and the Limits of the Inevitability Claim The trustworthy-ML field the mechanism argument lives in, and an honest test of its load-bearing claim. Calibration is fixable, which strengthens the confidence-as-wrong-category point; and the mainstream view that hallucination is mitigable, not inevitable, is accommodated by decoupling the thesis from hallucination rate.
  7. Literature Review: Abstention, Deferral, and the Receiver The constructive thread in trustworthy ML. Selective prediction lets a model decline to answer; learning to defer makes that choice depend on the downstream expert's competence, not the model's own uncertainty. The ML field reached action-relative admissibility independently, but stopped at the model boundary, which is where this work begins.
  8. Literature Review: Supply-Chain Integrity and Its Limit The first of the two intertwined field surveys. Software supply-chain security is organized around transparency, validity, and separation. Its validity property establishes an unchanged, authorized artefact, not a correct or fit one, which leaves admissibility open in the field's own words.
  9. Literature Review: Where Integrity Becomes Physical The second field survey and the point of fusion. OT security inverts IT priorities (safety first) because its actions are physical and irreversible. It meets supply-chain integrity at the safety boundary, where the Triton case records an unverified diagnosis promoted to fact and acted on.
  10. Literature Review: The Safety-Case Exclusion, Defended The problem statement set aside the safety-assurance literature. This entry earns that exclusion by engaging it, and its hardest case, assurance cases for machine learning. The field structures an argument over evidence whose standing it presupposes, about a component at design time, and names the admissibility gap only to hand it on.
  11. Literature Review: The Provenance and Policy Fields Widening two rivals from their specifications to their fields. Provenance, from its founding survey, records origin and feeds a trust judgement made elsewhere; its ML forms gesture at use but only in a document for a human. Policy evaluates supplied rules over supplied attributes and presupposes the model this work builds.
  12. Literature Review: Trust, Reliance, and Automation Bias Grounding the premise that the next owner relies on conveyed state and acts on it. The trust-in-automation field has studied over-reliance (misuse, automation bias, blind reliance) for decades, with fatal aviation cases where crews acted on conveyed state indistinguishable from correct. Its remedy, calibrated reliance, requires basis, but is addressed to a human.
  13. Literature Review: The Handover, in Epistemology The bounded coda. The review ends where its problem is oldest: the epistemology of testimony, whose transmission-versus-generation debate is the epistemology of the handover, and whose standard example is the promotion of a claim beyond its basis. Philosophy has the case but no name for the elevation; this work supplies it and departs to the engineering.
  14. Literature Review: What Confidence Cannot Be Testing the wrong-category argument against ML's best uncertainty tools. Confidence decomposes into aleatoric and epistemic uncertainty (neither is basis); even distribution-free conformal prediction gives only a marginal guarantee, not a per-claim warrant. The whole apparatus answers a different question than admissibility.
  15. Literature Review: Where Attestable Sits in OT Security A positioning entry, not another OT survey. OT security is layered perimeter defence built on Purdue zones, and it asks one question: is this crossing authorised. Admissibility is orthogonal to it, asking whether a value's basis suffices to act on. Attestable is not a perimeter layer but a gate at the cyber-physical boundary the perimeter waves values through.
  16. Literature Review: What AI Governance Mandates, and Where It Stops The AI governance frameworks (NIST AI RMF, the EU AI Act, the OECD recommendation, algorithmic auditing) all locate transparency and accountability at the handover, requiring that basis reach the receiver, and the NIST generative profile even names confabulation and over-reliance as official risks. But they mandate that the basis arrive; they do not adjudicate whether a particular claim suffices for a particular action.
  17. Literature Review: When the Explanation Is an Approximation The rejoinder that explanation methods fill the gap fails on the field's own terms: interpretability is a proxy the field admits its metrics cannot certify, the dominant post-hoc methods are approximations that give false assurances, in deployment explanations serve internal engineers rather than affected users, and the flagship counterfactual remedy explicitly declines to convey the decision's logic. A receiver cannot recover admissibility from a post-hoc explanation.
  18. Literature Review: The Learned Perimeter OT segregation is increasingly enforced by machine learning: the boundary monitor is a learned intrusion detector or anomaly scorer, and the state estimate is guarded by ML false-data-injection detectors. But a learned perimeter emits a score, not a basis, and can be walked across by adversarial design (Random Forest and J48 accuracy fell 16 and 20 points under attack). Moving the firewall to ML relocates the admissibility gap; it does not close it.
  19. Literature Review: When the Model Makes the Decision When machine learning makes or shapes an OT control decision, every safeguard the field reaches for lands somewhere other than admissibility. Safe reinforcement learning constrains actions probabilistically but cannot warrant a single one; state estimation can be corrupted beneath the controller without tripping its checks; and the trust layer bolted on top is post-hoc explanation, which describes the model rather than justifying the irreversible act it triggers.
  20. Literature Review: The Uncertainty Apparatus, End to End Machine learning's full uncertainty machinery - MC-dropout, deep ensembles, evidential deep learning, out-of-distribution detection - each answers a distributional or novelty question, not whether a claim's basis suffices to act. A model asserts a class with 91% confidence about pure Gaussian noise. And the field's own comprehensive survey concludes that estimating predictive uncertainty is not sufficient for safe decision-making.
  21. Literature Review: The Calibration Reassurance, and When It Breaks Calibration is offered as the rescue of confidence, but it does not hold. The metric that certifies it (ECE) has numerous flaws; the headline result that modern nets are miscalibrated does not generalise to recent architectures (conceded, and it costs the thesis nothing); and calibration dissolves under dataset shift - models are confidently wrong about entirely out-of-distribution data. A fair-weather guarantee, and irreversible actions are not taken only in fair weather.