Y Combinator co-founder Paul Graham argued that token counts are becoming an inadequate way to measure AI inference as increasingly capable models can solve more complex problems with fewer tokens.
Graham Calls for New AI Inference Metric
On Tuesday, in a post on X, Graham questioned how AI inference is currently measured and priced.
"Although you pay for AI by the token, that’s not the unit of inference, because you get more problem-solving per token as models improve," he wrote.
He argued that the industry needs a standardized measure that focuses on the amount of problem-solving an AI system can accomplish rather than simply counting the tokens it generates.
"Someone needs to define the unit," Graham said, suggesting it could involve "a chain of increasingly hard problems, each pair of which can be solved by a single model."
AI Costs Beyond Tokens
Last month, an AlphaSense study found that higher-priced frontier AI models from OpenAI and Anthropic could deliver better results at lower overall costs than cheaper Chinese models on complex financial analysis tasks.
The study of 246 tasks suggested businesses should consider total task costs and output quality rather than token prices alone.
In June, Anthropic launched Claude Fable 5 at twice the price of Claude Opus 4.8, charging $10 per million input tokens and $50 per million output tokens.
The launch came as Wells Fargo & Co. (NYSE:WFC) strategist Ohsung Kwon warned that rising AI inference costs could pressure the AI investment trade, citing companies that had exhausted AI budgets within months.
He also flagged rising infrastructure costs and potential interest-rate increases as risks, with Nvidia Corp. (NASDAQ: NVDA) particularly exposed to any infrastructure spending pullback.
Chinese AI Token Usage Surged
In July, Chinese AI models captured a record 58% of tokens processed by U.S. firms on OpenRouter, nearly tripling their share since mid-January. Their usage had briefly reached 63% in early July, up from less than 10% at the start of 2025.
DeepSeek had led the surge, emerging as the most-used Chinese AI model among U.S. firms as developers increasingly adopted Chinese alternatives.
Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.
Image via Shutterstock
Login to comment