Checking the AI agent's month-end close took as long as doing it, because nobody had written the process down.
One of the first things I learnt in technology sales, from a manager in my second job, was that we don't know what we don't know. I have been talking about human in the loop for years on exactly that basis. Theory, and nothing empirical behind it. Which puts me in the same position as every LinkedIn post on the subject, and I would rather not be there.
Propeller Industries went and got the evidence.
The accounting firm wanted to know whether an AI agent could run a month-end close, the monthly exercise of reconciling every account so the books can be signed off. Most tests of this kind have no way of marking the result, because nobody knows what the right answer was. Propeller solved that by working backwards.
They took a real client's close that their own controllers had already completed, then rebuilt the books as they had looked before any of that work was done, with every bill entered and all the operational activity present. The completed close became the answer key. The agent was given the unfinished books and never shown the key. Mabela Zenullari, who ran the test, was explicit that the point was to find out how the agents would fail.
They didn't. The output held up, and the problem turned up somewhere else entirely.
Where it went wrong was the checking. A controller had to go through what the agent had produced and confirm it, and that took about as long as doing the close by hand would have taken. So the time saved on the work was spent again on the review, and the exercise netted out at roughly nothing.
No document existed setting out how each reconciliation should be performed, so the agent worked out its own route through every one. The routes were usually sensible and rarely the route a controller would have taken. That left the controller with nothing to compare the work against. They could not run down a procedure ticking off steps, because there was no procedure written down, so instead they had to reconstruct the agent's reasoning from the beginning and satisfy themselves that a correct answer had been arrived at correctly. Zenullari called it a second set of books to review.
Most people will read this and conclude that verifying AI work is expensive. That is true and it stops one step short of the useful part, which is what Propeller did next.
The agent improvised because there was nothing to improvise against, and that single absence caused both halves of the problem. It is why the agent picked its own route, and it is why the controller had nothing to hold that route up against. Propeller reports the review cost and what they did about it. My reading goes a step beyond that. The missing document is not overhead added to make supervision affordable, it is what the agent needed in order to work predictably in the first place, and only once it exists does checking become a comparison rather than a re-performance.
Think about what review actually is. You check one thing against another thing. In most organisations only the first is ever named, and the second lives in the reviewer's head, which works perfectly well for as long as the reviewer has time to consult it. Name the second one or you don't have a control.
The same absence shows up again the moment something goes wrong. When a result looked off, nobody at Propeller could say whether the agent had failed or had faithfully carried out a workflow that had never been documented. Every error is ambiguous. You cannot debug the tool and you cannot debug the process, because you cannot tell which one you are looking at.
Their response was to go and write the thing down, which meant handing each controller a set of general ledger accounts and the job of documenting the workflow end to end, what inputs are required, what steps are taken, in what order. Not a redesign of the reconciliation, and not a new template. A written version of what was already in their heads. Once a reconciliation exists as a written standard, the agent's work can be checked against that standard in minutes rather than against someone's judgment in hours.
The software company Glide reached the same conclusion from the other end, having already written most of its standard down before it started.
Glide published an account in August of rebuilding its engineering process so that AI agents write most of its production code. Humans still decide what gets built. The agents produce the changes, and those changes used to go to an engineer to read before being merged, which is the normal arrangement in software and is called pull request review.
That review had been one of the more valuable parts of an engineer's day. As the agents produced more code it turned into a queue to clear, and then to clear again tomorrow. What broke it is the opposite of what you would expect. The code was good enough and consistent enough that reading it stopped being a judgment and became a stamp, and the engineers themselves asked to review less, on the grounds that the code needed less scrutiny rather than more.
Rather than asking anyone to read faster, Glide spent months making the standard machine-checkable. It tightened its linters, which are tools that reject code breaking agreed rules, and built out its evals, which are repeatable tests scoring whether an agent's output is acceptable. It merged its separate code repositories into one, so an agent could see the whole system. It made feature flags the default, meaning code can ship to production while staying switched off, so releasing and exposing became two separate decisions. And every change now passes through a merge queue, tested against the current state of the codebase before it is let in.
None of those changes was the system, Glide says, they were the foundations that had to be true before the system could exist.
Only then did humans come out of code review. The reading is now done by other agents checking the work, and Glide reports that ninety per cent of changes go through with no person reading the code.
It is the same basic move Propeller made, expressed in tests and tooling instead of prose. Two judgments were deliberately kept with people. After a feature is built, a person asks whether it does what was asked. After an automated repair is proposed, a person asks whether the fix is safe to ship. Neither involves reading line by line. The machinery holds the quality bar and the humans hold intent and consequence.
Where I would push back on Glide is the evidence. They publish the ninety per cent, the deployment frequency and the cycle times. They do not publish a defect rate, an escaped-bug count, or incident data for the changes that shipped unread. So the conclusion that the control had become unnecessary is their own, made confidently, and unmeasured. Propeller's test had an answer key and Glide's has none, which is a real difference in what the two accounts are worth.
Neither is independently audited either, and both companies sell services alongside what they published. That is the evidence available and it is worth saying so.
Put side by side, the two cases turn on the same variable. Propeller had no written standard, so the work was improvised and the review cost as much as doing it over. Glide had built one into its tooling, and could then decide which parts of review still needed a person and which did not. The question was never whether a human sits in the loop.
Human in the loop, yes, of course. But there has to be a standard they can assess against, and somebody has to write it down before the agent starts work rather than after it finishes.
Sources
- Propeller Industries, Can an AI Agent Run Your Month-end Accounting Close?, August 2026
- Glide, Creating Slipstream: How we rebuilt our software engineering process for AI agents, Simon Fletcher, 6 August 2026
The Propeller and Glide accounts are published by the organisations that ran the systems described, in both cases alongside commercial material; no independent verification is claimed for either. The agent in Propeller's test was Claude, made by Anthropic. This article was drafted with Claude.