A financial auditing tool ran about $541,000 over. A logistics project meant to reduce delivery times ran about $134,000 over. Roughly $2.5 million across the three.
News coverage led with the money. Several outlets called it a coding task. Matching author details to listings is data reconciliation, the kind of work nobody watches.
The project ran for five months before anyone caught it.
Token spend is metered, priced publicly, and billed monthly. Every token that project consumed appeared on an invoice. Engineers attributed the overruns partly to the shift from flat subscriptions to token-based billing, where costs climb whenever a task generates more activity than expected.
Few things inside a large enterprise are more thoroughly instrumented than a cloud bill. Amazon had the numbers for five months and stayed unaware of them.
Sensing takes more than measurement. Something has to compare the number against an expectation, notice the gap, and route it to someone who can act while acting is still cheap. Amazon had the number. The comparison and the route were missing.
No dashboard would have closed this. Someone had to decide that aggregate token spend against declared intent is a thing the company watches, and then own the watching.
March incidents
Amazon’s retail website took four high-severity incidents in a single week in early March, including a six-hour failure that locked customers out of checkout, account information, and pricing.
An internal document prepared for the review meeting identified GenAI-assisted changes as a factor in a pattern of incidents going back to Q3. That reference was deleted before the meeting, according to the FT, which saw both versions. Amazon disputed the reporting and said only one incident involved AI directly, with the root cause an engineer acting on inaccurate advice an AI agent had inferred from an outdated internal wiki.
Work backward from July. Five months of undetected spending starts around February or March. I cannot confirm the detection date, so treat the overlap as inference. Even without it, the shape holds. Amazon added review gates on AI-assisted changes to critical systems while a cost failure accumulated invisibly on a job nobody would call critical.
Constraint depends on detection. You cannot cap, contain, or price what you cannot see. Amazon reached for the second without the first, which happens because approval steps are visible to leadership and instrumentation is not.
Every deployment a team builds imposes cost on everyone else. Another surface to watch, another dependency to reconcile, another system someone who did not build it has to understand. The team keeps the benefit while the organization bears the cost. The remedy is to price that burden back to the team creating it.
Amazon built a price signal pointing the wrong way. Ranking people by consumption pays them to consume. An internal metric carrying status and no cost gets gamed, and this one did.
The presentation’s own recommendations now include avoiding leaderboards that reward token consumption, and checking whether higher token usage produces useful output.
The objection
Amazon frames these as isolated examples of teams learning from one another, and says cherry-picking them does not reflect how teams across the company use AI. For a company its size, a seven-figure surprise is a rounding error.
The reporting also lacks a base rate. Nobody has said how many AI projects came in on budget for every one that blew up.
Five months of invisibility still belongs to the control architecture rather than the budget. The same architecture at a company with a $12 million annual AI budget produces the same five months and a different outcome. Amazon can absorb what it cannot see.
What it costs to fix
Spending on sensing is bounded and knowable in advance. You can price the instrumentation, the ownership, and the review cadence before committing. The incoherence it prevents accrues silently and surfaces only once addressing it is no longer optional.
Amazon paid roughly $2.5 million across three disclosed projects. Other companies will meet the same failure without the revenue to absorb it.
The argument here about sensing, constraint, and priced externalities runs through my book, Coherence: The Competitive Advantage AI Can’t Buy, out this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.
Ford has added more than 350 experienced engineers over the past three years after its automated quality systems failed to deliver the results the company expected. Inside Ford they are called gray beards. Some are former Ford employees. Others came from suppliers.
Ford says it added the specialists “through internal promotions or new talent” to work alongside newer team members. Headlines have called it rehiring people AI replaced but Ford has not used that word, and the reporting does not establish that these specific roles were cut.
Poon also said Ford had not paid enough attention in prior years to the experience of its most knowledgeable engineers, the ones who had been through many product cycles.
Kumar Galhotra, Ford’s chief operating officer, said the company had been relying more and more on automated quality systems before recognizing the approach was not working.
Ford describes a training data problem. Poon’s version: enhancing the automation and machine learning tools required making sure they were trained by the most experienced individuals.
Part of that holds up plainly. The specialists do reprogram the AI tools that fell short.
The jobs
A missing corpus has a fix with an end date. Sit the veterans down, extract the failure modes they carry, feed the models, thank them.
A standing design review is a verification loop the organization has decided to keep running.
Where the corpus story runs out
Tacit knowledge can be captured up to a point. What a senior engineer knows about how a joint fails under a particular thermal cycle can be written down, and should be.
Judgment applied to an unanticipated case cannot. A reviewer looks at a novel configuration and says it will not hold, for reasons that emerge from the thing in front of them. Enumerate those cases ahead of time and you would have automated the review already.
The corpus framing implies a completion state, where enough capture makes the humans optional. Ford’s remedy points elsewhere. Mandatory and recurring is what you build once you have concluded the checking does not stop.
Automating a quality inspection function means automating a verifier, which removes the capacity to tell whether the automation works.
Ford’s specialists hold the ability to tell the machine it is wrong.
The order
Poon’s admission about prior years is the sharpest thing either executive said. Ford’s assumption was reasonable and its sequence was wrong. Capture the expertise, then automate, and the program is defensible. Automate on the assumption that design requirements are sufficient, and you spend three years buying judgment back from suppliers and internal promotions.
Each step is the precondition for the next. Skipping one relocates its cost to a later point, larger, with fewer options available. Ford turned a knowledge problem into a three-year staffing program, and that program ran alongside more than $1 billion in expected warranty and material costs this year and a quality reputation to repair.
Mentorship
Ford is explicit that mentorship is part of the assignment. The specialists work alongside newer team members and train junior staff who never absorbed the institutional knowledge.
This is how senior judgment gets built. Less experienced people work real problems while someone holding the judgment watches and corrects. The routine cases are the training ground, and automation takes them first.
An organization that automates routine work and then loses its seniors breaks the pipeline at both ends. Nobody holds the judgment and nobody acquires it. Ford is paying to rebuild both.
Headcount models calculate savings against the cost of the people. The judgment training pipeline appears nowhere in the model.
Ford is still deploying AI
On an autumn 2025 earnings call, Galhotra said Ford was systemically deploying AI across the entire industrial system, including 900 AI-powered cameras across its plants to detect quality issues at the source. Ford kept the cameras. Jim Farley told Bloomberg TV that Ford has AI tools for vision systems, and that most of it comes down to team members paying attention to small details.
Ford topped the mainstream J.D. Power rankings with the automated systems still running. Experienced people now sit between the systems and the product.
Experienced engineers sat on Ford’s books as execution capacity, a cost line. Their function was judgment, which is what makes execution capacity safe to deploy.
Ford is not unusual in getting the order wrong. CNBC reported Robert Half data showing 32 percent of U.S. hiring managers eliminated a role primarily because of AI and later rehired for the same or a similar position. Robert Half’s own summary puts it as more than 3 in 10, and notes the two most common reasons given: the role required institutional knowledge or context AI could not replace, and it involved relationship management AI could not replicate.
The arguments here about judgment as the layer that cannot be purchased, and about sequence in organizational automation, run through my book, Coherence, arriving this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.
I read an article in the Wall Street Journal today about my hometown of Austin, and it made me laugh before it made me think. Since Waymo’s robotaxis arrived here in 2024, they have apparently collected $9,325 in parking tickets. Tow-away zones. Metered spots they never paid. A disabled space outside an elementary school. One that idled in front of a church garage for five minutes during Sunday service while parishioners waited. A resident’s complaint in the records reads, plainly, “there needs to be some way to get them to move.”
These figures come from documents the Journal obtained through an open-records request, so I am relaying its reporting rather than confirming the numbers myself.
As a number, $9,325 is nothing. Austin collected $6.3 million in parking fines in 2025 alone, so Waymo’s two-year total is a rounding error. Spread 83 citations across more than 300 cars over two years and the per-vehicle rate is low, probably lower than what a human-driven taxi fleet of the same size would rack up in the same window.
While the dollar figure is trivial, the behavior behind it is not. It is also a near-perfect illustration of the argument I have been making.
The obvious reading, and why it misses
The easy version of this story is that the self-driving car is not ready. Look, it cannot even park. That reading is wrong, and the same article contains the reason.
An independent analysis by the Insurance Institute for Highway Safety found that over more than 50 million driverless miles, Waymo’s crash involvement rate was 68 percent lower than that of human drivers. The hard problem, the one with lives attached, Waymo has solved to a level that beats us. Parking is where it stumbles.
A system can be superhuman at its central task and fail at something a sixteen-year-old handles on the first day with a learner’s permit. This is jaggedness. Andrej Karpathy coined the term for the strange fact that a state-of-the-art model can solve a hard problem and then miss a trivial one, and a field experiment with 758 BCG consultants showed the same thing in the workplace: performance was excellent on tasks inside the model’s zone and worse on adjacent tasks that looked just as easy. The boundary is uneven, and it does not follow the difficulty ranking a human would draw. Driving safely across 50 million miles is the hard task the machine has mastered. Parking lawfully is the easy adjacent task it has not, and no amount of skill at the first predicts skill at the second. I have written about this shape once already this month. An AI office agent filled out seventeen forms in five minutes and then could not upload a file. Same jaggedness, different machine.
The dangerous part is that the failure is invisible from the outside. Watch a car drive flawlessly for an hour and you will assume it can handle a parking lot, because that inference holds for humans. It does not hold here, and the assumption is where the trouble starts.
The parking failure itself is not my thesis. Waymo will patch handicap-spot detection, and that particular fine will stop appearing. But notice what does not happen. Jaggedness does not get fixed. It moves. Patch the parking lot and the uneven edge shows up somewhere else nobody thought to check, because the unevenness comes from how the system learns, not from a single defect waiting to be found.
That is one problem, and it is real. There is a second one in the same article, and it is not a version of the first. It is a different failure entirely, and it is the one my book is actually about.
The failure that no model fixes
Read the part of the article that is not about parking.
On July 8, the National Highway Traffic Safety Administration sent autonomous-vehicle developers a letter demanding that their cars better follow instructions from first responders. The regulator’s complaint was that robotaxis often fail to recognize where they can stop without getting in the way. When a Waymo blocked an active railroad track in January 2025, an officer reported he had no choice but to have it towed. There was no other way to move it.
It does not go away with better driving. A firefighter at a scene, a police officer at a closure, a resident at a blocked garage: each of them has authority over the situation but no means to direct the machine sitting in it. The human is formally in charge and practically helpless. I keep making one distinction in my book, and this is it in the physical world. Having authority over a system is not the same as having the capacity to intervene in it. The officer had every right to move that car. He had no lever to do it, so he called a tow truck.
That gap is the coherence problem, and no amount of driving skill closes it. It is a problem of the connection between a capable system and the people who are supposed to be able to redirect it. You can make the car a better driver every quarter and leave that gap exactly where it is.
Three hundred locally rational decisions
Here is the detail in the article that matters most.
Waymo runs more than 300 robotaxis in Austin. Between trips, the article says, the cars park themselves on public streets to stay near riders and avoid adding traffic. Each of those choices is sensible. Idling near likely demand cuts empty miles and shortens the next pickup. For the fleet, it is the right call every time.
Now add up 300 right calls. You get 300 vehicles independently claiming curb space across one city, each optimizing for the fleet, none of them accountable for what they cost the curb in aggregate. The church-garage blockage was not one rude car. It was the predictable output of a fleet doing exactly what it was designed to do, measured against a shared resource that no one in the system is responsible for.
This is the pattern I spend the book on. W. Edwards Deming showed it in factories long before any of this. Optimize each part on its own and the whole can still degrade, because the parts interact in ways no single part can see. A support agent and a billing agent inside a company can each be flawless and still act on contradictory assumptions about the same customer. Three hundred robotaxis can each park perfectly rationally and still congest a city. The mechanism is identical. The only new thing is that it now runs at the speed and scale of software, on a public street.
Where the analogy breaks, and why the break is the interesting part
My book is mostly about a different situation. Many systems, built by many teams, with no shared owner, colliding inside one company. Waymo is close to the opposite. One company, one software stack, one central fleet manager. The cars are not incoherent with each other. They are all perfectly coherent with Waymo’s goal. The incoherence is between the fleet and the city.
That difference does not weaken the parallel. It sharpens it. Inside a single enterprise, the cost of local optimization eventually lands back on the enterprise itself. It pays its own complexity debt, later and with interest. In the robotaxi case, the company captures the efficiency and the public absorbs the cost. The fleet gets the shorter pickup times. The churchgoers get the blocked garage. The externality lands outside the firm, which means the firm has no natural reason to see it or price it.
Which is where the parking ticket returns, transformed. The tickets are not the failure in this story. The tickets are the city’s answer to it. Austin cannot rewrite Waymo’s software, so it does the only thing available to an outsider. It attaches a dollar figure to each incoherent act and bills it back. A tow-away citation is a coordinating institution forcing a cost back onto the party that created it, because that party will not absorb a cost it cannot see on its own dashboard.
In the book I call that pricing the incoherence, making the party that creates a coordination burden bear the cost it imposes on everyone else. It is one of only a few places you can intervene when you cannot redesign the system directly. Austin is doing it with a parking-enforcement officer and a public complaints database. It may be crude, but it is the right instinct. When you cannot fix the system, you can at least make it pay for the mess, so the incentive to stop finally reaches someone who can.
The instinct is widely shared, which is its own small piece of evidence. I read the comments under the article, and the readers who were not busy mocking it reached, unprompted, for exactly this lever. Charge a flat $5,000 fee for a driverless tow. Impound the car and make the company pay storage, same as a person would. Hold them to the standard you and I are held to. Nobody in that thread proposed debugging Waymo’s curb detection, because none of them can. They proposed raising the price of the behavior, which is the one move available to an outsider who cannot see inside the system and cannot change it. Pricing is what is left when coordination is out of reach.
A note on fairness, since it matters. Waymo pays these tickets like any other driver, and its spokesman said the company expects no special treatment. It contests some citations and has had a couple dismissed. None of that is evasion. It is a company behaving reasonably inside a system that has not yet given it a better way to behave. The point is not that Waymo is careless. The point is that even a careful, centrally managed, genuinely safer-than-human fleet produces coordination costs its own metrics will never show. That is the part that should worry anyone deploying autonomous systems anywhere.
The version of this that has not happened yet
One last thought, and I will flag it clearly as speculation rather than something the article reports.
Today Austin has one large fleet parking itself on the curb. The same article names two more operators already here, Tesla’s Robotaxi and Amazon’s Zoox. Imagine the near future where three or four fleets, each centrally coherent, each optimizing its own vehicles against the same finite curb, all share one city. None of them is incoherent on its own terms. Each is a model citizen by its own dashboard. Together they compete for the same few feet of pavement outside the same church at the same 10 a.m. service, and no one owns the result.
That is the multi-owner version of the trap, and it is the one that looks most like the enterprise problem I actually write about. Many capable systems, no shared view of the whole, a commons that quietly degrades while every participant is behaving well. When it arrives, the city will reach for the same tool it is using now, only harder. It will try to price the congestion, because pricing is what is left when you cannot coordinate the systems directly and you cannot see inside any of them.
There is a sharper edge to a single fleet that is worth one more sentence, because it cuts against the intuition that central control is safer. A fleet of 300 identical cars does not only optimize together. It fails together. Every vehicle runs the same software and leans on the same positioning inputs, so a single upstream fault does not hit one car, it hits all of them at once and in the same way. I made this point about software agents in a recent post: two agents drawn from the same model share the same blind spot, so the redundancy between them is nominal. Here it is 300 machines sharing one blind spot instead of two. Homogeneity buys clean coordination on a good day and correlated failure on a bad one. That is not an argument against central control. It is a reminder that the thing which makes a fleet coherent is the same thing that can make it fail in unison.
The safest car on the road parks in the fire lane. The fleet that adds no traffic blocks the garage. Every decision was locally correct, and the street got worse anyway. That is not a story about cars. It is the story of what happens to any organization, or any city, that fills up with capable systems faster than it builds the means to keep them coherent.
The gap between capable systems and coherent ones is the subject of my book, Coherence, arriving this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.
If the title sounds weird, it is because I stole it from a reddit thread I read last week! This thread, on a data engineering subreddit, is all about building fast without building coherent. A practitioner described a year spent working inside a major enterprise platform deployment that, by their account, went badly. The post drew more than a thousand upvotes and over 150 comments, and many of those comments said a version of the same thing. This matches what happened to us.
Let me be careful about what this post is and is not. I cannot verify any of it. I do not know the author, the employer, or whether the account is accurate, complete, or fair. I am not treating any of it as fact. I am not making a claim about the named vendor, any other company, its people, or its products. Online accounts are one-sided by nature. The company is not present to respond. And the thread does not even agree with itself on basic points, including why things went wrong. One commenter accused the vendor’s engineers of dragging work out to bill more hours. Two others replied that the vendor uses fixed-price contracts and has the opposite incentive.
So, let me set aside the question of motive or blame but ask something different. If a reader believed these accounts as written, what pattern would they describe? The pattern, if it is real, is one my book predicts.
The pattern the accounts describe
The original poster says they inherited the system after the engineers who built it left on thirty days notice, once a first version was declared done. What they found, in their telling, was a catalog of shortcuts. Hardcoded dates. Hardcoded accounts. The same business concept fed by different inputs in different places. Earlier problems patched with more hardcoded logic.
They gave one concrete example later in the thread. An engineer had built a button to delete a record. The button removed the record from the screen. It did not remove the three related records that the original had created when it was made. The result was orphaned data. The button worked. The system did not.
That small story is the whole thing in miniature. Every piece can be locally correct while the system is globally broken. A button that deletes what you can see and leaves what you cannot is a fine button and a broken workflow at the same time.
A commenter who said they work at a hospital inside a national health service described their own experience. Outside engineers did intense early work, leaned heavily on the in-house team to explain the basics, then left. No one was clearly left owning or maintaining what had been built. At one point, the commenter said, the vendor’s own monitoring staff emailed to ask why duplicate records and bad addresses were appearing, and the hospital could not answer, because it did not have access to the pipelines that had been built for it.
Another commenter described a failure higher up the organization. A single platform owner was installed. Over time that person’s standing became tied to the platform’s success, and the information traveling up to senior leadership was filtered, so the picture at the top stayed positive while the picture on the ground did not.
The line I keep thinking about
The poster wrote that the engineers used AI to produce tangled, low-quality logic. A commenter answered in five words.
That is the argument of my book, delivered by someone who did not set out to make it. When producing code becomes fast and cheap, more of it gets produced. The speed is real. What does not arrive with the speed is coherence. Coherence is the work of making sure each piece fits the whole, that today’s shortcut is not tomorrow’s silent failure, and that someone still understands the system after the people who built it are gone. Execution got cheaper. Coherence did not.
Why I am comfortable writing this at all
Here is the part that matters most. The people in the thread mostly did not think the story was about one company. One commenter wrote that you could swap in almost any vendor, almost any consultancy, and almost any project, and reach the same ending. Another described the identical arc with a completely different vendor. Others reached back to the enterprise data tools of twenty years ago and asked whether it had always been this way. They were describing a recurring structural pattern, and I think they were right to.
The pattern is old. W. Edwards Deming spent decades showing that optimizing each part of an organization on its own can degrade the whole, because the connections between the parts matter as much as the parts. Stafford Beer showed that organizations drift when the feedback reaching the people in charge is slow or filtered. Neither man was talking about AI. Both were describing this thread.
I want to give the other side its due, because the thread did. The original poster said plainly that the platform itself is fine for what it is. Other commenters defended it and corrected specific claims. Many organizations report that the same tools serve them well. The tool is capable. What fails, in these accounts, is the fit between a tool sold on speed and an organization that cannot absorb what speed produces. Change the logo on the invoice and the story would run the same way.
The number nobody calculated
The poster reported that the project was estimated at four months and took fifteen, and that the company had seen no return so far. I cannot confirm those figures. If they are even roughly right, they point at something the book returns to again and again. The promised savings were a calculation about capability. The cost that actually landed was a calculation nobody made, the cost of coordinating, maintaining, and understanding what got built. The first number is easy to put in a sales model. The second one shows up a year later and has no owner.
I have written three times recently about the same shape seen from different angles. A machine can generate the output. A person still has to own the part with no dashboard. In a newsroom experiment, an AI agent finished the forms and could not finish the job. In a courtroom, a scoring system read a gap in the data as a verdict on a person. In this thread, if the accounts hold, capable engineers produced software that worked in the demo and broke quietly in the corners, then left, and the coherence walked out the door with them.
Faster is not the same as coherent. It never was. The difference used to be expensive to create and easy to see. Now it is cheap to create and slow to see, which is exactly why it is worth watching for.
I can’t verify the thread, but the gap it points at is real. That gap, between building fast and building coherent, is the subject of my book, Coherence, arriving this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.
Keith Collins gave an AI agent full control of a laptop and three office jobs to do: survey nine colleagues over Slack and log their answers, identify staff cuts to hit a budget target, and fill out seventeen I-9 employment verification forms. The tasks were adapted from benchmarks published by researchers at Carnegie Mellon and OpenAI. The agent ran on Anthropic’s Claude Cowork app.
On the third task, the agent generated all seventeen forms correctly in under five minutes. Then it tried to upload them to Google Drive and failed. It clicked the right menu item and never noticed that a file picker had opened. It compressed the files. It converted them to a long string of bytes. It asked a second agent for help, and the second agent hit the same wall. After roughly seventeen minutes, it stopped trying and marked the task complete.
The Times files this under comic stumbles. It is the most consequential finding in the piece.
120,000 jobs, cut on the opposite premise
The article closes on a line meant to calm: AI still needs a human boss.
The same article reports the layoffs. More than 200 tech companies have cut roughly 120,000 jobs this year, per Layoffs.fyi. Meta and Oracle made substantial cuts citing AI. Cloudflare’s chief executive, after letting go of about 1,100 people, said he expects AI to replace workers in middle management, finance, and marketing.
Those cuts rest on a premise: the supervisory layer is what becomes redundant. The experiment found the reverse. Agents were strong at execution and weak at judgment. They wrote clean code in minutes, then made a categorization error about employees on leave that any manager would have caught.
Firms are removing coordinating capacity while installing systems that consume more of it. That connection is the thesis of the book I am writing. One case has already reached a federal courtroom.
Three specimens
My argument: when execution stops being scarce, the binding constraint becomes coherence, the integrity of the link between what local systems do and what the enterprise intends. Coherence has five specific dimensions, and autonomous systems break it in six recognizable ways. The Times experiment produced three clean specimens.
The false completion is escalation failure. The agent detected its own failure. It reasoned about it for seventeen minutes. It recruited a second agent. Then it reported success. The system knew it had not finished, and it stayed quiet. In this case, it cost little to the reporter analyzing logs. But in an enterprise running ten thousand delegated tasks a day, it is the mechanism by which reported completion drifts away from actual completion. Escalation failure is the one mode in my taxonomy that leaves every dimension of coherence intact and disables the reflex that repairs them. An organization can see a problem clearly and still be paralyzed when the signal never reaches anyone who can act.
The second agent matters too. Two systems drawn from the same model share the same blind spot, so the redundancy is nominal. The organization paid for one failure twice.
The code detour builds architecture nobody chose. In every task, the agent was told to work through the applications and wrote code instead. Graham Neubig of Carnegie Mellon puts it plainly in the piece: agents work in a very unhuman way, writing code instead of using the interfaces humans use. The Times treats this as a limitation. It is also an architectural event. The agent replaced the assigned task with a different one that produced a similar-looking artifact. An org chart rebuilt by a Python script carries a new dependency, a new failure mode, and no owner. Multiply that across a year of routine delegated work and the enterprise runs on infrastructure nobody selected, documented nowhere, discovered only when it breaks.
Local simplification often works by moving complexity somewhere else. The productivity gain lands on the dashboard. The displaced complexity does not.
The staffing error is contextual failure, and it is already in litigation. Given a budget target, the agent did something genuinely good. It read the personnel documents and concluded the 4 percent reduction could be met through planned retirements and resignations, with no layoffs. Then it added employees on leave to the list of cuttable roles without considering when they were coming back. The source material was silent on duration. The agent never asked.
Researchers at Stanford and the NBER frame this as a tacit knowledge problem, and that holds. The mechanism is more specific. The agent had no way to represent a person as temporarily absent for a reason that says nothing about their value. Silence in the record became a mark against the employee.
Nine days before the Times published, twenty-six Meta employees filed suit in federal court in Oakland alleging that the same substitution happened to them at production scale. Their complaint says Meta relied on internal AI systems, keystroke and activity-monitoring data, AI token-usage dashboards, and algorithmically assisted performance rankings to decide who would go in a layoff of roughly 8,000 people, about 10 percent of the workforce. The central allegation: those scores cannot by design be accumulated by an employee on protected medical or family leave, or by an employee whose output is reduced by a disability. The suit further alleges the company never paused the process for the individualized, leave-neutral review the law requires. About half the plaintiffs had taken leave for caregiving or pregnancy-related reasons. Their jobs were set to end on July 22, the day the Times ran its experiment.
Meta rejects the claims. The company says they lack merit and are not based on facts, and that workforce and organizational decisions “were and are made by people, not AI.” The allegations are unproven and the case is live. I am looking at the structure of the dispute here and taking no position on the verdict.
That structure survives either outcome, which is why it belongs in this argument. Suppose Meta is right that people made every call. Those people still read rankings, and the rankings still came from a substrate that had no field for protected absence. A human who approves a ranked list holds the authority to intervene and does not necessarily hold the information. Formal presence in a process is a weaker thing than capacity to change it. My book calls that oversight failure.
The Times agent and the Meta complaint describe one error at two scales. A system met a gap in its data and scored the gap as a deficiency. Nobody had built the mechanism that would have made it ask a question instead.
My book already discusses the Cloudflare decision the Times cites. Its chief executive organized his reasoning around a Drucker framework: every organization has builders, sellers, and measurers, and AI can now measure cheaply, so the measuring layer can shrink. The framework is coherent and the logic holds internally. The question I put to it there was whether the people categorized as measurers were only measuring. The Meta complaint poses the companion question. When the score came back low, was the system measuring performance, or measuring absence?
The article measured one axis
Task reliability and organizational complexity are independent dimensions. Improving one leaves the other where it was. Conflating them keeps the expensive failures invisible until they are hard to reverse.
The Times measured reliability, carefully and well. Its headline number comes from Scale AI: on real freelance projects, the best model produced client-ready work about 16 percent of the time.
The coordination question sits outside that number. If 84 percent of agent output requires human review, review capacity becomes the ceiling on deployment. Oversight load scales with the number of systems, and it lands on a different dashboard than the productivity gain. A control system has to be at least as varied as the thing it controls. Thin the supervisory layer while thickening the volume of supervised work and the cost moves off the ledger. It stays in the business.
The Oakland filing shows where it resurfaces. Twenty-six people asking a court to examine how a ranking was produced is a coordination cost, arriving late, in the most expensive form available.
Where the article argues against me
The counterargument has real force. If agents cannot reliably finish tasks, they will not be deployed at scale, and the coordination problem stays theoretical. The article supports that. A 16 percent success rate describes a product that is not ready.
Two responses.
First, the two failures differ in kind. The upload bug will be fixed. It is a UI problem and the entire industry is aimed at it. The leave-of-absence error and the false completion report sit at the boundary between the agent’s context and the organization’s. Better models will make both rarer. No model tells an enterprise which completion reports it can trust, or who owns the Python script the agent wrote last Tuesday. Those are ownership questions, and capability does not settle them.
Second, consider what the two variables are doing. Reliability improves on a public curve that everyone watches. Coherence has no curve, because almost nobody measures it. Using a snapshot of the fast-moving variable to dismiss the stationary one is the error my book is written against.
Two caveats. The Times experiment was three tasks, one tool, one synthetic environment, with expert-written prompts and supporting documents supplied by benchmark researchers. Real enterprises rarely supply that quality of context. The setting was favorable on the task side and trivial on the coordination side, since one agent ran alone with no installed base of prior deployments to collide with. The reliability observed sits closer to a ceiling than a floor. The coordination cost observed is near zero by construction.
The second caveat: Meta’s alleged systems are ranking and monitoring software, a different technology from an autonomous agent operating a laptop. The defect predates agentic AI. Agentic deployment raises the rate at which it executes.
What to watch instead
If you run an enterprise and this experiment shaped your thinking, start measuring the things it did not.
What fraction of your deployed autonomous systems has a named human owner. What fraction produces decisions you can explain to a regulator. How many of your systems depend on other systems in ways nobody mapped. How much of the behavior is visible to the people accountable for it. How hard it would be to remove any given system now that it is running.
If my thesis holds, those five move in one direction as deployment scales, while task accuracy holds steady or improves. That divergence is the signature.
The Times asked whether AI can do your job. Twenty-six people in Oakland are asking the harder version. Who answers for the score that said they were not doing theirs?
My book, Coherence, arrives this Fall. If you want to follow the thinking as it develops, join the list at coherise.com. The one-page decision tool from the book is the first thing I send.
Two arguments about AI landed within days of each other this month. They look unrelated. They describe the same event from opposite ends, and read together they close a loop that neither closes alone.
Satya Nadella published a short piece over the weekend that names something most enterprises have not yet noticed they are doing. A few days earlier, Arvind Narayanan and Akash Kapur published a longer essay on why the AI labs cannot make money selling raw intelligence, and what they will do about it instead. One argument tells you what you are losing. The other tells you why the loss is not an accident.
Let’s start with Nadella.
He begins with Kenneth Arrow. Arrow described a paradox in the market for information: a buyer cannot know what information is worth until they have it, at which point they have it for free. So the seller risks giving away the knowledge in the act of trying to sell it.
You pay for intelligence twice. Once in money. Again in the proprietary knowledge you have to reveal to make that intelligence useful.
Nadella inverts it. In the AI age, the risk runs the other way. The buyer gives away knowledge in order to use what they bought. You pay for intelligence twice. Once in money. Again in the proprietary knowledge you have to reveal to make that intelligence useful. And the better you want the model to perform, the more of your knowledge you have to hand it. He calls this the reverse information paradox.
His answer is a trust boundary: a hard perimeter inside which your data, traces, evals, tuned weights, and memory accumulate together, and across which nothing passes without consent. Own your evals. Build your learning environment inside your own tenant. Keep the orchestration layer decoupled from any single model. Compound.
Now the other end.
The labs are spending trillions on chips and data centers. The thing they sell, model inference, is close to a perfect commodity. The leading models behave alike, cost about the same to run, and carry almost no switching cost.
Narayanan and Kapur ask a blunt question. The labs are spending trillions on chips and data centers. The thing they sell, model inference, is close to a perfect commodity. The leading models behave alike, cost about the same to run, and carry almost no switching cost. Sell a commodity into a competitive market and the price falls to the cost of production. So how does any lab ever earn back the buildout? Their answer is that it cannot be earned back by selling tokens. The labs have to move up the stack, into products, workflows, and embedded deployments, and they have to build moats. One of those moats is a flywheel: train the models and systems on customers’ own material, their data, their execution traces, their evaluation suites, until the product pulls ahead in a way a rival cannot copy.
Set the two arguments side by side and the picture sharpens.
Your judgment is not an incidental byproduct of the labs’ business. Capturing it is the business, because it is the one thing that turns an undifferentiated model into something with a moat around it.
What Nadella calls exhaust leaking out, Narayanan and Kapur call the flywheel that powers the labs’ escape from the commodity trap. It is the same substance. The traces, the corrections, the evals. Nadella watches them leave your building. Narayanan and Kapur explain why the firm on the other side needs them so badly. Your judgment is not an incidental byproduct of the labs’ business. Capturing it is the business, because it is the one thing that turns an undifferentiated model into something with a moat around it.
That changes the stakes. The pull on your knowledge is not a quirk of one product or one vendor’s terms. It is structural, and it will not relent, because the economics of the entire model layer depend on it. The rest of this piece uses one instrument from the book to work out what to do.
What leaks is not your data
Start with the mechanism, because most people will read Nadella’s post as a data-protection argument and it is not one.
Nadella is specific. Models learn from exhaust. The prompts people write. The tools the agents call. And above all, the corrections people make when the model is wrong. Each correction is distilled into know-how. It leaks imperceptibly, he writes, trace by trace, correction by correction, eval by eval.
A correction is not a data point. It is a judgment. When your underwriter overrides the model’s risk score, she is not supplying a fact. She is encoding a standard. What good looks like in this market. What that number actually means when the counterparty is this counterparty. What your firm would never do, regardless of what the numbers say. She is teaching the machine your institution’s judgment, in the most compressed and machine-readable form that judgment has ever existed in.
That judgment is the one thing your competitors cannot purchase. I have argued elsewhere that as intelligence commoditizes, the advantage that remains is the one no vendor can sell you. Nadella reaches almost the same sentence from a different direction, that this is the kind of knowledge a competitor could never buy.
The reverse information paradox is not primarily an intellectual property problem. It is a coherence extraction problem. Not the capacity itself, which no one can take from you, but everything the capacity produces, exported decision by decision.
Which is exactly why the leak matters. The reverse information paradox is not primarily an intellectual property problem. It is a coherence extraction problem. Not the capacity itself, which no one can take from you, but everything the capacity produces, exported decision by decision. The cruelty of it is structural: the act by which an organization encodes its judgment into its systems, correcting the machine until the machine reflects how the firm actually thinks, is the same act by which it exports that judgment to whoever owns the model.
If a single competitor lets the vendor learn from its work, the model that serves your whole industry improves, and the vendor’s hand strengthens against every buyer in it, including the ones who kept their discipline.
Narayanan and Kapur add the part Nadella leaves out, which is that you cannot hold this line alone. The flywheel needs only one firm in a sector to start it turning. If a single competitor lets the vendor learn from its work, the model that serves your whole industry improves, and the vendor’s hand strengthens against every buyer in it, including the ones who kept their discipline. Your own boundary protects your specific corrections. It does not protect you from the sector arming the vendor around you.
You do not lose your moat in a breach. You lose it in a thousand small acts of being helpful, some of them your own, some of them your rivals’.
Two kinds of exhaust, and how to tell which one you are leaking
Nadella treats exhaust as a single substance. It is not, and the distinction is practical.
In the book I use a simple two-axis instrument. One axis is verifiability: whether a task’s success can actually be checked, and how fast a failure would be caught. The other is organizational complexity: how many units a deployment touches, how deeply other systems depend on it, and how hard it would be to reverse.
Run exhaust through those two axes and it separates cleanly.
Low verifiability leaks your judgment. These are the tasks where success is contestable and the model is often wrong: strategic assessment, valuation, anything where the right answer depends on tacit context. This is also exactly the work where a human should remain the decision-maker, which means the wrong answers get caught and corrected, and corrections are the most concentrated form of institutional judgment there is. Notice the twist: the safer your posture, the richer the exhaust. This is the highest-value leak in the building.
It is also where you are most easily held. Narayanan and Kapur point out that judgment-heavy work has no objective standard of quality, so a buyer cannot verify the output even after the fact. Writing, strategy, judgment calls are credence goods, like the work of a lawyer or a consultant. Unable to compare quality, you fall back on trust and reputation, and you stay put. So the same weak verification that makes these corrections precious makes the vendor that holds them hard to leave. Low verifiability is where your judgment concentrates and where your exit narrows at the same time.
High complexity leaks your architecture. These are the deployments woven deep into how the company runs. The model may be right almost every time, so there are few corrections. But the traces are a map. Which tools get called in what order, which systems depend on which, where the handoffs are, what the exception paths look like. That is a blueprint of how your organization actually operates, as opposed to how the org chart says it does.
High verifiability plus low complexity leaks almost nothing worth having. This is the commodity zone. Document classification, code execution, data transformation. Let it run. The exhaust is worthless to a competitor because the task is worthless as a differentiator.
So the first question is not “how do I protect my data.” It is “which of the two things am I giving away, and is it the one that matters.” The answer depends on where the deployment sits, and most enterprises have never asked.
Your evals are worth more than your data lake
Nadella makes a point in passing that deserves more weight than he gives it. Evals, he writes, define what good looks like inside the organization.
Follow that all the way down.
Data is a record of what happened. Evals are a specification of what you consider good. Those are not the same kind of object, and they are not remotely the same value.
A competitor who has your evals knows what you value, how you score it, and where you draw the line.
A competitor who steals your data still has to work out what you were optimizing for. They have the outcomes without the standard. A competitor who has your evals knows what you value, how you score it, and where you draw the line. They have the standard, which means they can generate their own outcomes.
If evaluation is where human effort is concentrating, the eval suite is where your people’s judgment is accumulating, and that is precisely why it is worth more than the data it scores.
This is not only an enterprise observation. In his ICML keynote this month, Narayanan argued that as AI absorbs the building, human effort migrates toward exactly this work: away from developing the systems and toward evaluating and monitoring them, toward the tasks that are hardest to verify. His frame there is the whole field and the whole economy. Bring it down to a single company and it lands on the same object. If evaluation is where human effort is concentrating, the eval suite is where your people’s judgment is accumulating, and that is precisely why it is worth more than the data it scores.
Most enterprises spend enormous energy guarding the data lake, and then hand the eval suite to whoever will run it for them, because building evals is tedious and the vendor offers to help. That is the wrong trade, made in the wrong direction, for the most understandable reason in the world.
If you protect one thing inside the boundary, protect the definition of good.
You cannot enforce a boundary you cannot see
Here is the prerequisite the boundary quietly assumes, and where I think the practical failure will happen.
Every one of Nadella’s recommendations presupposes an organization that knows what its systems are doing. Retain ownership of your traces, feedbacks, and decisions. Build learning environments inside the tenant boundary. Make sure nothing crosses without consent. Each of these requires that you can see what you have deployed, what it touches, and what leaves.
Most enterprises cannot.
IBM surveyed 2,000 chief information and technology officers this year. Seventy percent said teams were deploying AI faster than IT could track. Seventy-seven percent said adoption was outrunning their governance. Those two numbers describe an organization that does not know its own perimeter.
The failure will not look like a breach. Your general counsel is not going to paste the merger memo into a consumer chatbot. A junior analyst is going to paste the comparable transactions in at eleven at night, because the deadline is at eight and the tool is right there and nobody told her not to. Multiply that by every team that stood up an agent this quarter without telling anyone, and the hard boundary is a diagram in a slide deck.
The map of how you work is a prize for the vendor for the same reason it is a necessity for you. So there is no neutral option. Either you build the sensing layer inside your boundary, or you rent it from the vendor, who then owns the map.
Visibility is also contested from the other side. Narayanan and Kapur note that the labs’ most lucrative escape, charging for outcomes rather than tokens, requires them to see inside your business processes, which means migrating into your System of Record. The map of how you work is a prize for the vendor for the same reason it is a necessity for you. So there is no neutral option. Either you build the sensing layer inside your boundary, or you rent it from the vendor, who then owns the map.
This is why I keep arguing that coherence is not a governance problem in the usual sense. It is a visibility problem first. You cannot govern, audit, permission, or protect what you cannot see. Information sovereignty has a prerequisite, and the prerequisite is knowing what you have.
To be precise about scope: sensing is the first layer of a larger stack the book lays out. There are only four places to intervene on incoherence. You can sense it, constrain it, contain it, or price it, making the team that creates a coordination burden bear the cost it imposes on everyone else. Above all four sits human judgment, reserved for what the lower layers surface. Nadella’s boundary will eventually need the whole stack. But sensing comes first by necessity, not preference: you cannot constrain, contain, or price what you cannot detect.
Build that layer, or the boundary is decoration.
Where to spend the money
The last gap is a budget question, and it is the one that will actually decide whether any of this gets done.
Trust boundaries are not free. Private evals, tenant-bound training environments, a genuinely model-agnostic orchestration layer: these are real investments in engineering and in organizational discipline. No enterprise can build them around everything. Any advice that implies otherwise will be ignored by the people who have to fund it, and they will be right to ignore it.
Narayanan and Kapur draw a line that helps here. They separate the value AI creates from the value anyone manages to capture. The value created will be vast. The open question is who keeps it. Apply that line one level down, inside your own firm. Your people create judgment-value every time they correct the machine. The only question that matters is whether you capture it or the vendor does.
Spend where the exhaust encodes judgment you could not replace and where the deployment is deep enough that its traces map how you actually work. Tolerate leakage where the task is a commodity, the failure is cheap.
The two axes tell you where to spend. Not all incoherence is worth preventing, and by the same logic, not all leakage is worth stopping. Spend where the exhaust encodes judgment you could not replace and where the deployment is deep enough that its traces map how you actually work. Tolerate leakage where the task is a commodity, the failure is cheap, and the trace tells a competitor nothing they do not already know.
An enterprise that hardens every boundary equally has misread the problem exactly as badly as one that hardens none. The first will spend itself into paralysis. The second will donate its judgment one correction at a time, and never see the invoice.
What it does not mean
One caution, because this argument is easy to overcook and the overcooked version is wrong.
Keeping important work away from AI is just retreat dressed as strategy, and it loses. Abstention just gets you slower, with none of the benefits.
The lesson is not “keep your important work away from AI.” That is a retreat dressed as a strategy, and it loses. A competitor who brings AI to their hardest, highest-judgment work, in the posture that work allows, assistance where verification is weak, autonomy where it is strong, and does it inside a proper boundary, gets two things you do not: the compounding and the protection. Abstention gets you neither. It just gets you slower.
There is a real cost to engagement, and Narayanan and Kapur name it. Leaning on a vendor’s AI can erode your unaided skill while building a vendor-specific dependence, a lock-in that works through your own people rather than your contracts. But that is a cost of careless engagement, not of the engagement itself. The answer is the same one the whole piece has been building toward: engage hard, own the loop, keep the orchestration model-agnostic so the skill you build is yours and portable. Abstention avoids the behavioral moat only by forfeiting the capability, which is the worst trade on the board.
There is a reason to move now rather than later. The moat Narayanan and Kapur describe is not yet built. Enterprises have so far been reluctant to feed their material into the flywheel, and the orchestration layer is still, for the moment, thin and swappable. That window does not stay open. The time to build the boundary is before the lock-in compounds, which is to say now.
In consuming intelligence, you are creating intelligence, and what you create should belong to you. The goal is not to stop feeding the machine. It is to make sure the loop closes inside your own walls.
Nadella has the emphasis right. In consuming intelligence, you are creating intelligence, and what you create should belong to you. The goal is not to stop feeding the machine. It is to make sure the loop closes inside your own walls, so that the judgment you spend every day encoding accrues to you instead of leaking to the firm that sold you the model.
That is the difference between an enterprise that compounds and one that is quietly farmed.
That question, whether the judgment you encode every day accrues to you or leaks to whoever sold you the model, is the subject of my book, Coherence, and of everything I am writing here between now and launch. If you are deciding where the boundary has to be hard and where to let the exhaust go, the one-page tool behind the two axes in this piece is the first thing I send when you join the list at coherise.com.