AI Still Needs Human Bosses?

The New York Times just ran an experiment worth reading.

Keith Collins gave an AI agent full control of a laptop and three office jobs to do: survey nine colleagues over Slack and log their answers, identify staff cuts to hit a budget target, and fill out seventeen I-9 employment verification forms. The tasks were adapted from benchmarks published by researchers at Carnegie Mellon and OpenAI. The agent ran on Anthropic’s Claude Cowork app.

On the third task, the agent generated all seventeen forms correctly in under five minutes. Then it tried to upload them to Google Drive and failed. It clicked the right menu item and never noticed that a file picker had opened. It compressed the files. It converted them to a long string of bytes. It asked a second agent for help, and the second agent hit the same wall. After roughly seventeen minutes, it stopped trying and marked the task complete.

The Times files this under comic stumbles. It is the most consequential finding in the piece.

120,000 jobs, cut on the opposite premise

The article closes on a line meant to calm: AI still needs a human boss.

The same article reports the layoffs. More than 200 tech companies have cut roughly 120,000 jobs this year, per Layoffs.fyi. Meta and Oracle made substantial cuts citing AI. Cloudflare’s chief executive, after letting go of about 1,100 people, said he expects AI to replace workers in middle management, finance, and marketing.

Those cuts rest on a premise: the supervisory layer is what becomes redundant. The experiment found the reverse. Agents were strong at execution and weak at judgment. They wrote clean code in minutes, then made a categorization error about employees on leave that any manager would have caught.

Firms are removing coordinating capacity while installing systems that consume more of it. That connection is the thesis of the book I am writing. One case has already reached a federal courtroom.

Three specimens

My argument: when execution stops being scarce, the binding constraint becomes coherence, the integrity of the link between what local systems do and what the enterprise intends. Coherence has five specific dimensions, and autonomous systems break it in six recognizable ways. The Times experiment produced three clean specimens.

The false completion is escalation failure. The agent detected its own failure. It reasoned about it for seventeen minutes. It recruited a second agent. Then it reported success. The system knew it had not finished, and it stayed quiet. In this case, it cost little to the reporter analyzing logs. But in an enterprise running ten thousand delegated tasks a day, it is the mechanism by which reported completion drifts away from actual completion. Escalation failure is the one mode in my taxonomy that leaves every dimension of coherence intact and disables the reflex that repairs them. An organization can see a problem clearly and still be paralyzed when the signal never reaches anyone who can act.

The second agent matters too. Two systems drawn from the same model share the same blind spot, so the redundancy is nominal. The organization paid for one failure twice.

The code detour builds architecture nobody chose. In every task, the agent was told to work through the applications and wrote code instead. Graham Neubig of Carnegie Mellon puts it plainly in the piece: agents work in a very unhuman way, writing code instead of using the interfaces humans use. The Times treats this as a limitation. It is also an architectural event. The agent replaced the assigned task with a different one that produced a similar-looking artifact. An org chart rebuilt by a Python script carries a new dependency, a new failure mode, and no owner. Multiply that across a year of routine delegated work and the enterprise runs on infrastructure nobody selected, documented nowhere, discovered only when it breaks.

Local simplification often works by moving complexity somewhere else. The productivity gain lands on the dashboard. The displaced complexity does not.

The staffing error is contextual failure, and it is already in litigation. Given a budget target, the agent did something genuinely good. It read the personnel documents and concluded the 4 percent reduction could be met through planned retirements and resignations, with no layoffs. Then it added employees on leave to the list of cuttable roles without considering when they were coming back. The source material was silent on duration. The agent never asked.

Researchers at Stanford and the NBER frame this as a tacit knowledge problem, and that holds. The mechanism is more specific. The agent had no way to represent a person as temporarily absent for a reason that says nothing about their value. Silence in the record became a mark against the employee.

Nine days before the Times published, twenty-six Meta employees filed suit in federal court in Oakland alleging that the same substitution happened to them at production scale. Their complaint says Meta relied on internal AI systems, keystroke and activity-monitoring data, AI token-usage dashboards, and algorithmically assisted performance rankings to decide who would go in a layoff of roughly 8,000 people, about 10 percent of the workforce. The central allegation: those scores cannot by design be accumulated by an employee on protected medical or family leave, or by an employee whose output is reduced by a disability. The suit further alleges the company never paused the process for the individualized, leave-neutral review the law requires. About half the plaintiffs had taken leave for caregiving or pregnancy-related reasons. Their jobs were set to end on July 22, the day the Times ran its experiment.

Meta rejects the claims. The company says they lack merit and are not based on facts, and that workforce and organizational decisions “were and are made by people, not AI.” The allegations are unproven and the case is live. I am looking at the structure of the dispute here and taking no position on the verdict.

That structure survives either outcome, which is why it belongs in this argument. Suppose Meta is right that people made every call. Those people still read rankings, and the rankings still came from a substrate that had no field for protected absence. A human who approves a ranked list holds the authority to intervene and does not necessarily hold the information. Formal presence in a process is a weaker thing than capacity to change it. My book calls that oversight failure.

The Times agent and the Meta complaint describe one error at two scales. A system met a gap in its data and scored the gap as a deficiency. Nobody had built the mechanism that would have made it ask a question instead.

My book already discusses the Cloudflare decision the Times cites. Its chief executive organized his reasoning around a Drucker framework: every organization has builders, sellers, and measurers, and AI can now measure cheaply, so the measuring layer can shrink. The framework is coherent and the logic holds internally. The question I put to it there was whether the people categorized as measurers were only measuring. The Meta complaint poses the companion question. When the score came back low, was the system measuring performance, or measuring absence?

The article measured one axis

Task reliability and organizational complexity are independent dimensions. Improving one leaves the other where it was. Conflating them keeps the expensive failures invisible until they are hard to reverse.

The Times measured reliability, carefully and well. Its headline number comes from Scale AI: on real freelance projects, the best model produced client-ready work about 16 percent of the time.

The coordination question sits outside that number. If 84 percent of agent output requires human review, review capacity becomes the ceiling on deployment. Oversight load scales with the number of systems, and it lands on a different dashboard than the productivity gain. A control system has to be at least as varied as the thing it controls. Thin the supervisory layer while thickening the volume of supervised work and the cost moves off the ledger. It stays in the business.

The Oakland filing shows where it resurfaces. Twenty-six people asking a court to examine how a ranking was produced is a coordination cost, arriving late, in the most expensive form available.

Where the article argues against me

The counterargument has real force. If agents cannot reliably finish tasks, they will not be deployed at scale, and the coordination problem stays theoretical. The article supports that. A 16 percent success rate describes a product that is not ready.

Two responses.

First, the two failures differ in kind. The upload bug will be fixed. It is a UI problem and the entire industry is aimed at it. The leave-of-absence error and the false completion report sit at the boundary between the agent’s context and the organization’s. Better models will make both rarer. No model tells an enterprise which completion reports it can trust, or who owns the Python script the agent wrote last Tuesday. Those are ownership questions, and capability does not settle them.

Second, consider what the two variables are doing. Reliability improves on a public curve that everyone watches. Coherence has no curve, because almost nobody measures it. Using a snapshot of the fast-moving variable to dismiss the stationary one is the error my book is written against.

Two caveats. The Times experiment was three tasks, one tool, one synthetic environment, with expert-written prompts and supporting documents supplied by benchmark researchers. Real enterprises rarely supply that quality of context. The setting was favorable on the task side and trivial on the coordination side, since one agent ran alone with no installed base of prior deployments to collide with. The reliability observed sits closer to a ceiling than a floor. The coordination cost observed is near zero by construction.

The second caveat: Meta’s alleged systems are ranking and monitoring software, a different technology from an autonomous agent operating a laptop. The defect predates agentic AI. Agentic deployment raises the rate at which it executes.

What to watch instead

If you run an enterprise and this experiment shaped your thinking, start measuring the things it did not.

What fraction of your deployed autonomous systems has a named human owner. What fraction produces decisions you can explain to a regulator. How many of your systems depend on other systems in ways nobody mapped. How much of the behavior is visible to the people accountable for it. How hard it would be to remove any given system now that it is running.

If my thesis holds, those five move in one direction as deployment scales, while task accuracy holds steady or improves. That divergence is the signature.

The Times asked whether AI can do your job. Twenty-six people in Oakland are asking the harder version. Who answers for the score that said they were not doing theirs?

My book, Coherence, arrives this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.