Skip to content
Paula Livingstone writing · projects · tools

Attestable Review: Working Notes

Literature Review: When the Model Makes the Decision

When machine learning makes or shapes an OT control decision, every safeguard the field reaches for lands somewhere other than admissibility. Safe reinforcement learning constrains actions probabilistically but cannot warrant a single one; state estimation can be corrupted beneath the controller without tripping its checks; and the trust layer bolted on top is post-hoc explanation, which describes the model rather than justifying the irreversible act it triggers.

The companion entry followed the OT perimeter as it turned into a learned classifier and showed the admissibility gap moved rather than closed. This entry goes further into the plant, to where machine learning does not merely watch the boundary but makes the decision: a model that sets a control action, schedules an intervention, or supplies the estimate an action is taken on. This is the case the thesis exists for, a probabilistic output driving an irreversible physical act, and the OT literature is now building exactly these systems. Read on their own terms, they confirm the problem from the inside and, in their most advanced form, reach for the very remedy the review has already shown to be insufficient.

When the controller is learned, safety is not guaranteed

Deep reinforcement learning is being adopted to control power and energy systems directly, replacing hand-built control schemes with a learned policy that maps the system's state to a control action. The critical review of this trend states the difficulty without hedging: while these algorithms have significantly advanced control schemes, they often fall short in guaranteeing safety, which is a critical concern in the operation of critical infrastructure like power and energy systems (Bui et al., 2024). The review frames the whole safe-reinforcement-learning subfield as a response to that shortfall, noting that in critical infrastructure safety issues always receive top priority while the learned controller may not meet the operators' safety requirements. This is the OT priority inversion, safety and availability before all else, now colliding with a decision-maker that cannot certify the safety of the action it selects. The learned controller emits an action; it does not emit a demonstration that the action is safe to take.

The formal machinery built to address this makes the limit precise rather than removing it. The technical survey of safe reinforcement learning casts the problem as a constrained Markov decision process, in which safety requirements are expressed as constraints alongside the reward the policy maximises (Kushwaha et al., 2026). This is a genuine advance, but a constrained policy bounds the probability or the expected value of a constraint violation; it does not furnish a per-action warrant that this action, now, is admissible. In an OT setting where exceeding a limit even once can be catastrophic, a guarantee that is distributional or in-expectation is the wrong shape, in exactly the way the uncertainty-quantification entry argued conformal coverage was the wrong shape: it is a statement about the policy's long-run behaviour, not about the basis of the single action about to move a valve.

The decision the model does not make but shapes

Learned models also enter OT decisions one step back, by supplying the estimate a controller acts on. A power-system controller acts on a state estimate, and that estimate can be corrupted at source: false-data-injection attacks affect the internal state-estimation process and can be built to bypass conventional bad-data detection, so the manipulated estimate is passed downstream without being flagged (Irfan et al., 2023). This is the admissibility problem in its purest OT form. The value the controller treats as the state of the world is not guaranteed to correspond to the state of the world, the corruption can be crafted precisely so the existing checks do not fire, and the control decision inherits the defect silently. Whether the downstream controller is classical or learned, it acts on a conveyed value whose basis it never sees.

The remedy that reaches for an approximation

The most telling sources are the ones that recognise the trust problem and try to solve it, because they reach for precisely the instrument the review has already weighed and found wanting. As machine learning is adopted for maintenance decisions in increasingly consequential settings, the survey of explainable predictive maintenance observes that because these methods are adopted for more serious and potentially life-threatening applications, the human operators need to trust the predictive system, and it is this need that draws in explainable AI to introduce interpretability into the prediction (Cummins et al., 2024). The move is understandable and it is the wrong move, for the reason the AI-governance entries established at length: a post-hoc explanation is an approximation of the model's behaviour, not evidence that a particular prediction's basis suffices for the action it triggers. The state of the art makes this concrete. A recent framework has a deep reinforcement learning agent emit compressor setpoints, a direct physical control action, and wraps it in a three-step explainability pipeline of input-perturbation testing, gradient sensitivity, and feature attribution, presented as delivering trustworthy control (Bezold et al., 2025). It is a careful and honest piece of engineering, and it is the exact pattern this thesis diagnoses: a probabilistic decision about an irreversible action, made trustworthy in name by a plausible explanation of how the model behaved, rather than by a representation of whether the decision's basis warrants the act. The field is building the wolf and the sheep's clothing in one system, in good faith, because the missing category has no name and no mechanism.

What this establishes

When machine learning makes or shapes an OT decision, every safeguard the field has reached for lands somewhere other than admissibility. Safe reinforcement learning constrains actions probabilistically but cannot warrant a single one; state estimation can be corrupted beneath the controller without tripping its checks; and the trust layer bolted on top is post-hoc explanation, which describes the model rather than justifying the act. This is the thesis's problem observed in the OT field's own most advanced work, not asserted from outside it. It is also the sharpest possible motivation for the contribution: these systems do not need a better controller or a better explanation, they need a representation, carried with the decision, of whether its basis suffices for the irreversible action, so that the elevation of a learned output into an executed command can be refused when it does not. That representation is what the perimeter, the estimator, and the explanation all leave out, and it is what this work builds.

References

Bui, V.-H., Das, S., Hussain, A., Hollweg, G. V. and Su, W. (2024). A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications. arXiv:2409.16256. arxiv.org/abs/2409.16256

Irfan, M., Sadighian, A., Tanveer, A., Al-Naimi, S. J. and Oligeri, G. (2023). False Data Injection Attacks in Smart Grids: State of the Art and Way Forward. arXiv:2308.10268. arxiv.org/abs/2308.10268

Kushwaha, A., Ravish, K., Lamba, P., Kumar, P. and Mahajan, A. (2026). A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety. arXiv:2505.17342. arxiv.org/abs/2505.17342

Cummins, L., Sommers, A., Bakhtiari Ramezani, S., Mittal, S., Jabour, J., Seale, M. and Rahimi, S. (2024). Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunities. IEEE Access. arXiv:2401.07871. arxiv.org/abs/2401.07871

Bezold, V., Wagner, P., Hofmann, J., Huber, M. and Sauer, A. (2025). Trustworthy and Explainable Deep Reinforcement Learning for Safe and Energy-Efficient Process Control: A Use Case in Industrial Compressed Air Systems. arXiv:2512.18317. arxiv.org/abs/2512.18317