🔍 Read the full analysis: How Engineers Can Look Past Token Counts To AI Outcomes on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Amazon published engineering guidance on Oct. 27 saying software engineers should be judged by outcomes, not token counts or AI-tool usage. The memo says usage metrics can help teams spot patterns, but using them as targets can raise costs and complexity without proving that customers or the business benefit.
Amazon published new guidance for its software engineers on October 27, warning managers not to treat AI token use or tool adoption as measures of job performance. The memo says those figures can help track current patterns, but engineers should be evaluated on their impact on customers, the business and their teams—a distinction that matters as companies try to measure whether AI tools produce useful results rather than activity alone.
The memo was written by Terk Terkowitz, a vice president and distinguished engineer in Amazon’s stores business, and presents a revised role guideline for software engineers created by the company’s most senior engineers. It says long-term outcomes are the right focus, while acknowledging they are often slow and difficult to measure. Some teams have used proxy measures such as token usage, AI adoption, code changes and code reviews.
Amazon calls those proxies “deeply flawed” when treated as targets. The memo warns that rewarding token use can conflict with the company’s frugality principle, while rewarding lines of code can work against simplifying software. It says such targets can increase costs and system complexity without showing whether the work produced better results.
The guidance does not say usage data is useless or that teams should stop using AI. It says those measures currently “correlate well” with important outputs, while cautioning that correlation does not make them suitable performance goals. Managers are told to examine both low and high adoption: understand why some teams use AI less, and check whether heavy users are delivering more. The memo also says engineers should document cases where current AI tools do not help, describing that as “a finding, not a failing.”
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
From AI Activity to Engineering Results
The memo addresses a measurement problem facing employers adopting generative AI: tool usage is easy to count, but value is harder to establish. A token total can show that a model was used; it cannot, by itself, show that a customer received a better service, that a task was completed faster, or that the resulting code is easier to maintain. Amazon’s guidance draws a line between using such data to investigate work and using it to rank employees.
That distinction matters for engineers as well as managers. A quota or leaderboard can reward activity even when an AI tool adds no value, potentially encouraging unnecessary prompts, code or review work. Amazon’s stated approach instead asks leaders to judge the work and understand where the tools help or fail. The memo also calls on leaders to use AI on their own production work and explain examples where it helped and where it did not.
The company’s broader AI investments provide a separate reason to scrutinize outcomes, but they do not prove that the new guidance was prompted by any particular investment result. The source article cites reporting that Amazon moved many flagship Nova models to maintenance and points to the paper value of its Anthropic stake. Those are distinct developments, with their own reporting and financial context; the memo itself focuses on how engineering work should be assessed.
Popular right now
Samsung 42-Inch S90H OLED 4K Smart TV Bundle
KitchenAid Artisan 5-Quart Tilt-Head Stand Mixer, Almond Cream
Ninja Foodi 10-Quart DualZone 6-in-1 Air Fryer, Black, DZ302
As an affiliate, we earn on qualifying purchases.
How Amazon Reached the New Guidance
The memo follows earlier reports of internal efforts to track AI use. The Financial Times reported in May that some Amazon employees assigned unnecessary work to an internal agent tool called MeshClaw to increase token counts. That account, including claims about adoption targets and leaderboards, was based on employee reports and is not independently established by the memo.
One related tracker is on the record: at the end of May, Amazon confirmed to Business Insider that a leaderboard called KiroRank had been “deprecated.” The company described it as an informal tool created by a group of employees and said it was not meant to encourage AI use for its own sake. The source article also quotes senior vice president Dave Treadwell telling staff, “Please don’t use AI just for the sake of using AI.”
The October guidance formalizes a broader principle for engineering roles rather than prescribing a fixed list of AI tools. The memo says specific AI definitions could quickly become outdated, so it anchors expectations in work that should remain relevant: impact on customers, the business and the team.
“If your work does not benefit from current AI tooling, that is a finding, not a failing.”
— Amazon engineering memo, published October 27
How Teams Will Apply the Guidance
The memo sets out principles, but the available material does not specify how individual teams will measure customer or business impact, how managers will weigh those results in performance reviews, or whether Amazon will replace existing dashboards with another reporting system. It also does not provide companywide data showing whether high AI adoption has led to better outcomes.
The earlier accounts of employees inflating token counts and of specific adoption targets come from media reporting, including anonymous employee accounts; Amazon’s memo does not confirm those details. Nor does the guidance establish that AI use is beneficial for every engineering task. Its stated position is that teams should examine the evidence and explain where current tools do or do not help.
Evidence From Teams and Results
The next test will be how Amazon’s engineering leaders put the guidance into practice: whether they review work at both low and high levels of AI adoption, and whether they connect tool use to measurable effects on customers, the business and team performance. The memo asks leaders to try the tools on their own production work and describe a task where AI helped and one where it did not.
Amazon was scheduled to report its third-quarter results on October 29, two days after the memo’s publication. Those financial results may provide broader company context, but they will not, on their own, show whether AI usage targets have improved engineering outcomes. Further evidence would require specific measures and reporting that connect AI-assisted work to results.
Key Questions
What did Amazon tell engineers about AI token counts?
Amazon’s October 27 memo says token counts and other usage measures should not be treated as performance targets. It says engineers should be judged by their impact on customers, the business and their teams.
Does the memo tell Amazon teams to stop using AI?
No. The memo says usage measures currently correlate with important outputs and does not call for stopping AI adoption. It warns against rewarding use for its own sake and asks teams to judge whether the tools improve results.
What does Amazon mean by “tokenmaxxing”?
In this report, “tokenmaxxing” refers to using AI tools in ways that increase token consumption or improve usage rankings without a useful work outcome. Amazon’s memo says that approach conflicts with frugality.
What should engineers do if AI tools do not help with their work?
The memo says they should document and share why the tools did not help. It describes that conclusion as “a finding, not a failing.”
What remains unknown about the new guidance?
The available information does not explain exactly how Amazon teams will measure customer or business impact in reviews, or whether existing usage dashboards will change. The memo also gives no companywide results proving that higher AI adoption improves outcomes.
Source: ThorstenMeyerAI.com
