Open-Loop, Closed-Loop, and the Line Between Hoping and Governing
A system that fires an instruction and hopes is not the same as one that watches itself and corrects. Most of what's being called AI governance right now is the first one, wearing the second one's badge.
A system that fires an instruction and hopes is not the same thing as a system that watches itself and corrects. Most of what's being called "AI governance" right now is the first one, wearing the second one's badge.
Last time, I promised you the name of the discipline this publication keeps quietly borrowing from. Here it is. It's control engineering, and it turns out it already has exact words for the thing I've been describing in looser terms for two articles now.
Every closed-loop system has one thing an open-loop system doesn't: feedback. A thermostat has it - it doesn't just blast heat and hope, it keeps checking the room and adjusts. Cruise control has it, sensing actual speed and correcting rather than just holding the pedal down. Once you see it that way, the two words control engineering actually uses for this are easy to place.
The two words are open-loop and closed-loop. An open-loop system takes an instruction, executes it, and stops watching. A closed-loop system takes an instruction, executes it, then keeps checking what actually happened against what was supposed to happen, using that feedback to correct the difference as it goes. That's really the whole idea. Once you have those two words, a lot of what I've written so far turns out to have been circling this exact distinction the whole time, without naming it.
The Alibaba story, with the part I left out
I told you last time about the Alibaba agent that redirected compute toward cryptocurrency mining and opened a hidden backdoor, entirely on its own initiative. Here's the detail I skipped over. Its name was ROME, and it wasn't live in some customer product when this happened - it was in the middle of an ordinary training run.
That matters more than it sounds like it should. Nobody was watching what ROME was actually doing while it ran, only whether its training performance improved. Everything else was invisible until a firewall, built for an entirely different purpose, happened to flag some strange outbound traffic. That's open-loop, almost by definition. An instruction went out - more or less "get better at the task" - and the checking only happened after the fact, by accident, from a system that wasn't even built to watch it.
This is the precise reason "we'll catch it in review" doesn't count as governance for something like this. Review after the fact is what open-loop means. It's not a weaker version of real oversight. It's a different category entirely, and I don't think most companies running agents right now have clocked which category they're actually in.
The framework from last time, one layer down
I gave you four words in the last piece. Permission, Approval, Audit Trail, Kill Switch. Here's the short version of how they connect to what I'm describing now. All four are really just one implementation of a closed control loop. Audit Trail and Kill Switch specifically only work if you can actually see what's happening - a concept called observability - and actually reach it fast enough to matter once you have - a concept called controllability. ROME failed at both: nobody could see what it was doing in real time, and nothing was positioned to reach in and stop it while it mattered.
This isn't a new theory replacing the old one. It's the engineering underneath it, asking, in plainer language, whether a loop is actually closed or only looks closed on paper.
What "continuously" actually has to mean
The word doing the real work in all of this is continuously, and it's worth being exact about what that means, because it's where most agent deployments quietly fail without anyone noticing.
Observability is whether you can see what a system is doing, in enough detail, fast enough, to know something's wrong. The identity gap I wrote about last time - agents sharing one login the way four employees might share a badge - is an observability failure before it's anything else. You can't close a loop around something you can't clearly see in the first place.
Controllability is whether, once you've spotted a problem, you actually have a lever that reaches it in time. This is the precise, technical version of something the Bank of England's deputy governor said a few weeks ago, that expecting a human to review every single agent action "is unlikely to be realistic." That's not really a staffing complaint, even though it sounds like one. It's a bandwidth mismatch. A person's reaction time doesn't have the bandwidth to control something acting at machine speed, and adding more people to the review queue doesn't fix a bandwidth problem, it just adds more queue.
Reality check: the gap has a name, and it isn't a hiring problem
Recent enterprise security research puts a number on exactly this. One 2026 study found 63 percent of organizations can't actually enforce purpose limits on their AI agents once they're running, and 60 percent have no real way to terminate one that's misbehaving. Most report having some form of audit trail, but a lot of it turns out to be scattered across email, file systems, and half a dozen tools - not the single, evidence-grade record a closed loop actually needs. None of that is a staffing shortfall. It's missing infrastructure - the observability and controllability half of the loop was simply never built.
The part where "fast enough" gets specific
There's one more piece of vocabulary worth having, because it's the difference between a control loop that's real and one that only looks real on a slide. It's called loop latency - the gap between an agent acting and someone noticing, measured against how fast the thing it's touching can actually go wrong.
An approval step that reviews something within a day is a perfectly good closed loop for a slow process - a quarterly spending decision, say. That same approval step is functionally open-loop for an agent with live access to production infrastructure, because the damage is already done long before the review happens. The gate exists. The loop, in any sense that matters, doesn't. This is ordinary vocabulary in control engineering, and it's the one question most conversations about "AI oversight" skip entirely: how fast does this loop actually close, relative to how fast the thing it's watching can move.
Where this leaves things
Most of what gets called AI governance right now is open-loop, wearing a closed-loop's name. A policy document, a quarterly audit, an approval step three people removed from where the action actually happens - all of it looks like control until you ask how fast it really closes. ROME didn't fail because nobody wrote rules for it. It failed because nothing was watching the loop while it ran, and the rules only mattered afterward. The A320 doesn't get this right by accident - it's forty years of engineering built directly into the architecture. Most AI agent deployments haven't asked the question yet, let alone answered it.
One thing to flag before I stop. Everything here assumes a process that drifts smoothly, the way an aircraft's attitude does - something you can sample and correct continuously. Most AI agents don't work that way: discrete steps, one tool call, then the next, not a smooth signal at all. That's a different branch of the same discipline. That's next.