AI Agents Are Not Ready to Run Critical Infrastructure

Model capability is not the main question

I would not put a model's capability first. As of November 2026, the main barrier to using AI agents in energy, water utilities and rail is primarily an engineering one: how to prove that every action is safe. NIST's Cyber AI Profile project connects AI risk management with IT, OT, industrial control systems and cybersecurity practices.

This is where things become uncomfortable. An agent with access to a dispatch system is not simply given a new tool; it gains a new path to a physical process. Prompt injection, poisoned telemetry, a compromised sensor or a malicious instruction for a tool can turn a plausible response into a dangerous command.

Hallucination is only one failure class. There are also outdated or conflicting data, sensor loss, rare operating modes, strict response deadlines and model behavior changes after an update. A standard language-model benchmark does not prove that a system will behave correctly during an emergency or an attack.

That is why the sensible architecture is still boring, and that is a compliment. The agent acts as an adviser, while deterministic safety logic checks whether commands are allowed. Tool access is restricted through allowlists and least privilege, networks are segmented, and a qualified operator confirms critical actions.

Tamper-evident logs are also needed for inputs, tool calls, approvals and executed commands. Without rollback, a safe state and manual control, autonomy becomes a bet that an unusual scenario will never occur.

A human in the loop does not solve accountability

A human in the loop does not automatically close the gap. During a flood of alerts, an operator may trust a confident recommendation, lack enough context or simply have no time to intervene. Oversight must therefore provide a real right to reject an action, understandable evidence and a rehearsed manual mode.

The legal gap is just as serious. After an accident, responsibility may be split among the infrastructure operator, integrator, model provider, software developer, contractor and a specific employee. Unless roles are fixed in advance, an investigation may end up debating whose failure it was rather than what caused it.

The boundary is fairly clear by sector. In power grids, an agent can help with forecasting and diagnostics; in water, with maintenance and process-quality analysis; and in rail, with scheduling, repair prioritization and incident review. But influencing protection settings, dosing, pressure, routing or movement authority requires assurances at the level of safety-critical components, not attractive accuracy on a general dataset.

My conclusion is simple: this is not about banning AI. It is about refusing unlimited authority without verifiable boundaries. The hardest component of such an agent may not be the model, but the accountability system around it.

We previously covered how Unicode homoglyph attacks can deceive AI agents into following phishing links or executing malicious commands. These attack paths are especially relevant when agents operate in critical infrastructure environments.