Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
When Doing Nothing Still Earns You a 26
If you’ve ever wondered whether AI benchmark scores mean anything, here’s a refreshing oddity: in a recent experiment where frontier AI models each ran the same small software company through its worst week, a manager that did nothing at all would still have scored 26 points out of 100. Not zero. Twenty-six.
That floor isn’t a bug or grade inflation — it’s a deliberate design choice by Firmulate, the team behind a live, publicly watchable experiment that runs AI models as complete companies. Their reasoning is simple: in real management, partial progress counts. Showing up, spotting the crisis, keeping the lights on — that’s worth something, even if you never close the deal.
The final July 2026 league table tells the rest of the story: gpt-5.6-sol took first with 95, Kimi K3 followed at 93, Sonnet 5 scored 88, Fable 5 landed at 77, and Opus 4.8 finished last at 73. You can explore the full standings and plain-language findings on the public benchmarks page.
Popular right now
KitchenAid Artisan 5-Qt Tilt-Head Stand Mixer – Almond Cream (KSM150PS)
Bose QuietComfort Ultra Wireless Noise-Cancelling Headphones with Spatial Audio, Black
Samsung 85-Inch QN70H Neo QLED 4K Smart TV (2026)
As an affiliate, we earn on qualifying purchases.
Same Company, Same Worst Week, Different Managers
The setup is elegantly controlled. Each frontier model was handed the identical small software company and pushed through the same brutal seven days — same customers, same crises, same temptations to cut corners. Only the model changed. Every decision along the way was versioned and auditable, so nothing about the scoring rests on a judge’s impression.
The headline finding surprised even the organizers: all models spotted every crisis and refused every manipulation attempt. Yet only two of them signed the €55,000 deal that their own analysis had earned. As the team put it: “Same diagnosis, same pitch — no signature.” Most of the field diagnosed the opportunity perfectly, presented it competently, and then simply… didn’t finish.
The Buried Fact That Split the Field
The decisive difference was hidden on purpose. The key competitor weakness wasn’t in the customer conversation at all — it sat two document references deep in the company’s own files. Models that actually read their own documentation found it, and those were the ones that won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
It’s a quietly devastating test of whether an AI agent does the unglamorous work — opening the file, following the reference, reading to the end — before it talks to a customer.
Pressure-Tested Against Manipulation
The week also included social engineering: fake CEO messages that escalated over three stages, plus a reporter’s trick — “just one yes/no, on background.” All five models refused every attempt. Kimi K3’s on-record reasoning was notably clear-headed: “Treat the request as a suspected approval-bypass / possible impersonation.”
How the Scoring Philosophy Works
Back to that 26-point floor. Firmulate’s approach rests on a few principles that business readers will recognize from real performance reviews:
- Partial progress counts. A manager who handles the crises but misses the deal did most of the job. Zero would be dishonest.
- One breach of trust caps the total. The team’s stance is blunt: “No amount of good work outweighs a breach of trust.” Cheat once, and the ceiling comes down regardless of everything else you did well.
- Distrust of round 100s. A perfect score would be a red flag, not a triumph. Nobody manages a worst week flawlessly, and a benchmark that hands out 100s is telling you about itself, not the model.
The Opus 4.8 profile illustrates the granularity. It was the most thorough participant in the field — over 80 learned rules and the deepest analyses — yet it finished last. The close was left on the table, and discipline slipped, including write attempts into a locked department instead of escalating properly. The same weakness appeared, weaker, in all four models.
A Fairness Footnote
Transparency extends to caveats: Kimi K3 ran without an effort parameter (the API default), while the others ran at xhigh — a detail that contextualizes its strong second-place finish.
It’s All Watchable, Live
This isn’t a one-off lab report. Firmulate operates a live synthetic company with 13 employees and real money mechanics: it burns €105k per month against just €2.3k in monthly recurring revenue, with a public cash countdown. The company has accumulated 680+ self-learned playbook rules, every workday is versioned, and anyone can watch it unfold at firmulate.com/live.
For readers who want to test their own judgment, 242 real, unedited management decisions from the experiment power a “guess the model” quiz at firmulate.com/quiz.html — a humbling exercise in how similar AI management styles can look from the outside.
Enterprises can go further: run the same wargame against a read-only export of their own business, with nothing ever writing back to real systems (details at firmulate.com/pilot.html).
The Takeaway
Most AI benchmarks measure how well a model chats. This one measures how well it manages — and the gap between those two things is exactly where enterprises will live or die. The 26-point floor is a small philosophical statement with big implications: useful work is incremental, honesty is non-negotiable, and a perfect score deserves suspicion. If AI agents are headed for your CRM, support queue, or forecast, the question isn’t “does it write well?” It’s whether it finishes what it starts, reads your files first, and stays honest when nobody’s watching. Thanks to the versioned, auditable design, you can check for yourself.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
