{"id":212,"date":"2026-08-02T19:45:17","date_gmt":"2026-08-02T19:45:17","guid":{"rendered":"https:\/\/coherise.com\/ideas\/?p=212"},"modified":"2026-08-02T19:45:17","modified_gmt":"2026-08-02T19:45:17","slug":"sensing-is-more-than-measurement","status":"publish","type":"post","link":"https:\/\/coherise.com\/ideas\/sensing-is-more-than-measurement\/","title":{"rendered":"Sensing Is More Than Measurement"},"content":{"rendered":"\n

The Financial Times reported this week on an internal Amazon presentation held July 28. Engineers walked staff through a set of AI cost overruns. The largest used Anthropic’s Claude Sonnet to match author details against product listings on Amazon’s retail site. It ran 860 percent over budget, cost $1.8 million, and never shipped<\/a>. Engineers said mistakes that were once trivially cheap had become “catastrophically expensive”<\/a>.<\/p>\n\n\n\n

A financial auditing tool ran about $541,000 over. A logistics project meant to reduce delivery times ran about $134,000 over. Roughly $2.5 million across the three<\/a>.<\/p>\n\n\n\n

News coverage led with the money. Several outlets called it a coding task. Matching author details to listings is data reconciliation, the kind of work nobody watches.<\/p>\n\n\n\n

The project ran for five months before anyone caught it.<\/p>\n\n\n\n

The data was there<\/h2>\n\n\n\n

A senior Amazon employee told the FT that it is difficult to figure out how much anything AI-related costs<\/a>. Said by someone at the company that runs the cloud everyone else buys AI on.<\/p>\n\n\n\n

Token spend is metered, priced publicly, and billed monthly. Every token that project consumed appeared on an invoice. Engineers attributed the overruns partly to the shift from flat subscriptions to token-based billing<\/a>, where costs climb whenever a task generates more activity than expected.<\/p>\n\n\n\n

Few things inside a large enterprise are more thoroughly instrumented than a cloud bill. Amazon had the numbers for five months and stayed unaware of them.<\/p>\n\n\n\n

Sensing takes more than measurement. Something has to compare the number against an expectation, notice the gap, and route it to someone who can act while acting is still cheap. Amazon had the number. The comparison and the route were missing.<\/p>\n\n\n\n

No dashboard would have closed this. Someone had to decide that aggregate token spend against declared intent is a thing the company watches, and then own the watching.<\/p>\n\n\n\n

March incidents<\/h2>\n\n\n\n

Amazon’s retail website took four high-severity incidents in a single week<\/a> in early March, including a six-hour failure that locked customers out of checkout, account information, and pricing.<\/p>\n\n\n\n

An internal document prepared for the review meeting identified GenAI-assisted changes as a factor in a pattern of incidents going back to Q3. That reference was deleted before the meeting, according to the FT, which saw both versions. Amazon disputed the reporting and said only one incident involved AI directly, with the root cause an engineer acting on inaccurate advice an AI agent had inferred from an outdated internal wiki<\/a>.<\/p>\n\n\n\n

Amazon’s response was a 90-day code safety reset across 335 critical retail systems<\/a> and mandatory senior-engineer sign-off on AI-assisted code from junior and mid-level engineers.<\/p>\n\n\n\n

Work backward from July. Five months of undetected spending starts around February or March. I cannot confirm the detection date, so treat the overlap as inference. Even without it, the shape holds. Amazon added review gates on AI-assisted changes to critical systems while a cost failure accumulated invisibly on a job nobody would call critical.<\/p>\n\n\n\n

Constraint depends on detection. You cannot cap, contain, or price what you cannot see. Amazon reached for the second without the first, which happens because approval steps are visible to leadership and instrumentation is not.<\/p>\n\n\n\n

KiroRank<\/h2>\n\n\n\n

Amazon ran an internal leaderboard called KiroRank that ranked employees by AI usage. Staff responded with what they called tokenmaxxing, deliberately inflating consumption to climb the rankings<\/a>. Amazon discontinued it.<\/p>\n\n\n\n

Every deployment a team builds imposes cost on everyone else. Another surface to watch, another dependency to reconcile, another system someone who did not build it has to understand. The team keeps the benefit while the organization bears the cost. The remedy is to price that burden back to the team creating it.<\/p>\n\n\n\n

Amazon built a price signal pointing the wrong way. Ranking people by consumption pays them to consume. An internal metric carrying status and no cost gets gamed, and this one did.<\/p>\n\n\n\n

The presentation’s own recommendations now include avoiding leaderboards that reward token consumption, and checking whether higher token usage produces useful output. <\/p>\n\n\n\n

The objection<\/h2>\n\n\n\n

Amazon frames these as isolated examples of teams learning from one another, and says cherry-picking them does not reflect how teams across the company use AI. For a company its size, a seven-figure surprise is a rounding error.<\/p>\n\n\n\n

The reporting also lacks a base rate. Nobody has said how many AI projects came in on budget for every one that blew up.<\/p>\n\n\n\n

Five months of invisibility still belongs to the control architecture rather than the budget. The same architecture at a company with a $12 million annual AI budget produces the same five months and a different outcome. Amazon can absorb what it cannot see.<\/p>\n\n\n\n

What it costs to fix<\/h2>\n\n\n\n

Spending on sensing is bounded and knowable in advance. You can price the instrumentation, the ownership, and the review cadence before committing. The incoherence it prevents accrues silently and surfaces only once addressing it is no longer optional.<\/p>\n\n\n\n

Amazon paid roughly $2.5 million across three disclosed projects. Other companies will meet the same failure without the revenue to absorb it.<\/p>\n\n\n\n

The argument here about sensing, constraint, and priced externalities runs through my book, Coherence: The Competitive Advantage AI Can’t Buy<\/a>, out this Fall. If you want to follow the thinking as it develops, join the list at\u00a0coherise.com<\/a>. The one-page decision tool from the book is the first thing I send.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

The Financial Times reported this week on an internal Amazon presentation held July 28. Engineers walked staff through a set of AI cost overruns. The largest used Anthropic’s Claude Sonnet to match author details against product listings on Amazon’s retail site. It ran 860 percent over budget, cost $1.8 million, and never shipped. Engineers said […]<\/p>\n","protected":false},"author":1,"featured_media":213,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-212","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-how-it-breaks"],"_links":{"self":[{"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/posts\/212","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/comments?post=212"}],"version-history":[{"count":1,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/posts\/212\/revisions"}],"predecessor-version":[{"id":214,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/posts\/212\/revisions\/214"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/media\/213"}],"wp:attachment":[{"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/media?parent=212"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/categories?post=212"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coherise.com\/ideas\/wp-json\/wp\/v2\/tags?post=212"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}