Guys, we are the in the mainframe era of AI hardware. People in the 60's thought computing was expensive too and the idea of having a computer on every desk, nevermind every pocket, nevermind every single piece of electronics in the world basically seemed like a complete pipe dream.
if you told someone in the 70's their toaster would have a supercomputer it in, they would think you were crazy. in 10 years your doorknob is going to have a local AI model it in.
This is computing 2.0 not the dot com bubble. 90% of inference will be at the edge in the future and there will still be super-computers and giant clusters doing cutting edge science and research, but for 90% of use cases youll just need a tiny local model, same reason you dont need a giant GPU in your smart tv.
The latest deepseek v4 pro model is 2-5x cheaper than Claude Sonnet 4.6. Cursor's Compose 2.5 that was just recently released is 6x cheaper than Sonnet.
The state of the art models are going to get better and more expensive and smaller models are going to get cheaper.
There will be a point where the intelligence of both the cheap and state of the art models are indistinguishable by humans like it is indistinguishable for me to understand the difference the difference between Terrance Tao and my university math professor.
I don't always need the smartest and most expensive models. I will need it every once in awhile and will gladly pay that price if I had to. What I do need is the model that will solve the current problem I have in a reasonable amount of time.
> The state of the art models are going to get better and more expensive and smaller models are going to get cheaper.
Why do you think this will be true?
Right now I see the major US labs betting on gaining an advantage from having way more compute, and I see Chinese labs competing with one another in a resource-scarce environment, so they place much more emphasis on compute-efficiency.
But the supply chains that feed into the massive data center growth in the US are strained; there are energy, memory, and logistical bottlenecks to name a few.
In the medium-long run, compute capacity will not grow exponentially forever. Somehow it has for decades, but there can be no infinite exponential growth, and that point may be when the planet really starts to cook itself.
Maybe the US labs will become more compute-constrained, and then have to compete on efficiency.
Or maybe things change fundamentally in some other way I'm not thinking of.
The labs have a perverse incentive to make things as expensive compute wise as possible. The only thing keeping this somewhat in check is competition, but it's intentionally being gatekept by locking up the supply of computing infrastructure. With 3 players it's pretty easy to collude even if indirectly. They can't burn trillions forever. Nvidia's 75% profit margins are not sustainable forever.
sorry to nitpick (I totally agree with what ur saying btw, I run Ministral-3b on my hardware as my go-to bc I don't usually need the "smartest and most expensive models")
I want open source models to fail for the most part,so llm fan boys have no other option than to go back to whatever it was they were doing before (crypto??).
I wonder how much of Uber blowing their AI budget and MSFT pulling their claude code licenses can be attributed to "tokenmaxxing".
When Meta announced token leaderboards and other followed, I could see this being the logical conclusion. That whole trend is so dumb because it leads to this.
Company announces they will measure developer performance by how many tokens they burn and constantly talks about how the best developers burn the most tokens. Developers see the message and start burning tokens. And then the company acts surprised when their bills go through the roof.
I personally use my OpenAI subscription pretty heavily, 2-3 agents running practically all day on various tasks but I never even get close to running into limits while I hear about others blowing through limits on multiple accounts in the same time period. I'm convinced that most of those folks and their elaborate workflows aren't really for productivity but for bragging rights about how much they use AI.
The same here, where I haven't come close to hitting any of my CC limits. Even though I'm more productive than I've ever been (as measured by finished, valuable tasks running in production) and I'm clearing out months of backlog, I have either one of two conclusions when I hear about others who suggest they need more:
1. I'm doing it wrong. Apparently I'm supposed to give it a vague paragraph about what the business does, and I can run off and sip margaritas and wake up to a fully fleshed business
2. They don't know what they're doing, and they're sending the LLM off on a wild goose chase that it does a reasonable job of working it's way out of, so they consider it success despite the waste.
> I'm convinced that most of those folks and their elaborate workflows aren't really for productivity but for bragging rights about how much they use AI.
This is quite the reductive, charged statement. Can I ask what subscription plan you're using?
My personal experience is unlike this at all-- I work on ever-expanding codebases so I can easily burn tokens. Not to mention, structured agentic coding with adverserial reviews & task organization is not token-efficient. Additionally, for the problems I'm working on, only xhigh or high reasoning gives me worthwhile results while saving time. There are definitely configurations where default consumption doesn't work.
For reference, I used 15 billion tokens (most of it cached) last month on my day job's enterprise plan. That doesn't include my personal plans' usage.
> I personally use my OpenAI subscription pretty heavily, 2-3 agents running practically all day on various tasks but I never even get close to running into limits
Same. But if I was working for an organization that measured token usage, you can bet I would be doing things like creating a cron job that uses claude to create a customized bespoke report update of the current status of all my open assigned tickets and message that to myself 4 times a day... token burn for zero purpose whatsoever.
I make like 2 prompts a week to gemini flash on the weband get more done than all the people that are exhibiting literal manic behavior in the way they use LLMs.
I really wish the management behind these dumb ideas couldnt just quietly pretend they never did it once it goes out of fashion.
The fact that somebody established a leaderboard for tokenmaxxing ought to follow you around like a black cloud for the rest of your career once the collective hallucination lifts and people realize just how monumentally stupid it was.
Alas they do all these stupid things together which makes it seem more defensible and then everybody forgets.
I seldom use my PC anymore ever since i got a laptop. with the cost per token increasing along with the random "features" where models will just eat through your tokens in one hour. I really have been tempted to turn my PC into a server to run local models on there
Insofar as I can tell, inference is on a certain path toward becoming "free". The models are now extremely powerful on high-end consumer hardware, and the efficiency trend seems likely to continue.
Here is a recent non-rigorous benchmark I ran against a bunch of models. Qwen3.6 35B A3B fine-tuned with opus data runs plenty fast on my local machine and produce outstanding results - easily in the top 5, comparable to GPT 5.5 Pro (which is $180/mtok).
I've predicted for years now that the industry will head down the path of the virus scanning vendors: selling subscriptions to be able to download the latest versions of models. I simply don't see how any other business model is remotely viable, except at the very highest end of inference or video gen.
Has this not been true for a long time now? Most companies have had enterprise/business level prices that was highly connected to usage for a what feels like at least a year.
kind of sobering to realize that whether your job can be profitably automated away comes down to what $/token some hyperscale AI provider can deliver… I suppose it’s nice that this article highlights some upward pressure on that number.
> Did we collectively forget second order thinking?
I bought 2x 16Gb NVIDIA cards this week because I don’t see hardware getting cheaper anytime soon, and because of that I totally don’t see the point of “waiting until prices go lower for graphics cards” because that might not for a long time yet!
In fact, if you include factoring in world events (and the ones that haven’t happened yet but eventually will e.g. China’s 2027 long planned take of Taiwan), then there’s no way graphics prices are going to be accessible to mere mortals until at least 2028.
But my real reasoning is that you’re going to see a flood of OpenAI and Anthropic users leave because of a) increasing pricing plans, and b) impeding business laws on the horizon about protecting sovereign data from AI (i.e data in cloud for training is a no no).
So what happens when people and companies one by one start leaving the SOTA AI cloud for from-good-enough-to-wow models? RAM and graphics cards become the new toilet paper, which is going to double again current prices.
Some of these coming price increases will move dev work back to dedicated shops and teams when individuals and non-devs won't want to pay the AI bill to finish and ship their projects.
An outside small dev shop or internal dev team can pay these prices and spread the cost over several customers or departments, but the era of giving everyone AI and telling them to dev stuff is about to be over.
This is only true if your world is limited to openai, antropic and alike.
There are a whole bunch of companies somewhere else in the world that are getting better and cheaper every month, hardware side included. all without the infinite VC money
It's hard to take this piece seriously if he's citing _Ed Zitron's_ math, and equally hard to make the blanket statement that flat-rate plans = "the current AI pricing". But yes, those pricing models were pretty silly and unsustainable.
Get back to me when there's an AI company that's actually profitable and we can compare their service and pricing.
Claiming that there's some small subset of their services (like inference per token) that's "profitable" doesn't mean anything when it relies on everything else that company is still paying for. If you could make money from it at current prices - why aren't they?
Otherwise it's just "how much they're willing to subsidize".
There is probably going to be a quarter or two of profits when the prices dramatically increase. Vibe coding techbros are hooked on the Iron Lung and may not want to get off.
At my work are multiple developers bragging about overnight AI usage to solve problems hands off. Yes they are wasting money and resources but the fad is here. People be vibe coding for now.
In like 6 months when all the costs need to be paid and the prices go up, we will see if these companies stay profitable. But I'm of the opinion that the vibe coding tech bros are more than enough to sustain a short or even medium term profit for these companies. Just on fad-energy alone (see OpenClaw)
------------
I dunno where this is all going. But I do have faith in human ingenuity still. Things are changing, possibly for the worse, but we need to make the best of it.
The worst of behaviors is wasteful and blatant fraud. There's something useful here though.
You are comparing two different model. It's like saying roadster is more expensive than model S. No model pricing actually increased, and I am using GPT-4o in the same price as it was before.
You can see price vs performance in artificial analysis and the the pareto optimal is all just 6 months old model.
You are incorrect. There, now we've both made unsupported assertions. Care to provide any evidence for your position?
For what it's worth, when I provide a Pangram link it's because I can already tell something is AI and I'm attempting to provide objective third-party confirmation so the conversation doesn't just degrade into me asserting that I have superior taste to you.
Guys, we are the in the mainframe era of AI hardware. People in the 60's thought computing was expensive too and the idea of having a computer on every desk, nevermind every pocket, nevermind every single piece of electronics in the world basically seemed like a complete pipe dream.
if you told someone in the 70's their toaster would have a supercomputer it in, they would think you were crazy. in 10 years your doorknob is going to have a local AI model it in.
This is computing 2.0 not the dot com bubble. 90% of inference will be at the edge in the future and there will still be super-computers and giant clusters doing cutting edge science and research, but for 90% of use cases youll just need a tiny local model, same reason you dont need a giant GPU in your smart tv.
This is where open source models are important.
The latest deepseek v4 pro model is 2-5x cheaper than Claude Sonnet 4.6. Cursor's Compose 2.5 that was just recently released is 6x cheaper than Sonnet.
The state of the art models are going to get better and more expensive and smaller models are going to get cheaper.
There will be a point where the intelligence of both the cheap and state of the art models are indistinguishable by humans like it is indistinguishable for me to understand the difference the difference between Terrance Tao and my university math professor.
I don't always need the smartest and most expensive models. I will need it every once in awhile and will gladly pay that price if I had to. What I do need is the model that will solve the current problem I have in a reasonable amount of time.
> The state of the art models are going to get better and more expensive and smaller models are going to get cheaper.
Why do you think this will be true?
Right now I see the major US labs betting on gaining an advantage from having way more compute, and I see Chinese labs competing with one another in a resource-scarce environment, so they place much more emphasis on compute-efficiency.
But the supply chains that feed into the massive data center growth in the US are strained; there are energy, memory, and logistical bottlenecks to name a few.
In the medium-long run, compute capacity will not grow exponentially forever. Somehow it has for decades, but there can be no infinite exponential growth, and that point may be when the planet really starts to cook itself.
Maybe the US labs will become more compute-constrained, and then have to compete on efficiency.
Or maybe things change fundamentally in some other way I'm not thinking of.
The labs have a perverse incentive to make things as expensive compute wise as possible. The only thing keeping this somewhat in check is competition, but it's intentionally being gatekept by locking up the supply of computing infrastructure. With 3 players it's pretty easy to collude even if indirectly. They can't burn trillions forever. Nvidia's 75% profit margins are not sustainable forever.
Things will normalize, but it will take time.
[delayed]
sorry to nitpick (I totally agree with what ur saying btw, I run Ministral-3b on my hardware as my go-to bc I don't usually need the "smartest and most expensive models")
> This is where open source models are important
open-weights, the training data isn't public
Deepseek V4 Flash is far cheaper still, and a better model to compare to Sonnet 4.6. I'm finding it a reliable workhorse.
Yep, people who never used it say it is not good.
I want open source models to fail for the most part,so llm fan boys have no other option than to go back to whatever it was they were doing before (crypto??).
I wonder how much of Uber blowing their AI budget and MSFT pulling their claude code licenses can be attributed to "tokenmaxxing".
When Meta announced token leaderboards and other followed, I could see this being the logical conclusion. That whole trend is so dumb because it leads to this.
Company announces they will measure developer performance by how many tokens they burn and constantly talks about how the best developers burn the most tokens. Developers see the message and start burning tokens. And then the company acts surprised when their bills go through the roof.
I personally use my OpenAI subscription pretty heavily, 2-3 agents running practically all day on various tasks but I never even get close to running into limits while I hear about others blowing through limits on multiple accounts in the same time period. I'm convinced that most of those folks and their elaborate workflows aren't really for productivity but for bragging rights about how much they use AI.
The same here, where I haven't come close to hitting any of my CC limits. Even though I'm more productive than I've ever been (as measured by finished, valuable tasks running in production) and I'm clearing out months of backlog, I have either one of two conclusions when I hear about others who suggest they need more:
1. I'm doing it wrong. Apparently I'm supposed to give it a vague paragraph about what the business does, and I can run off and sip margaritas and wake up to a fully fleshed business
2. They don't know what they're doing, and they're sending the LLM off on a wild goose chase that it does a reasonable job of working it's way out of, so they consider it success despite the waste.
> I'm convinced that most of those folks and their elaborate workflows aren't really for productivity but for bragging rights about how much they use AI.
This is quite the reductive, charged statement. Can I ask what subscription plan you're using?
My personal experience is unlike this at all-- I work on ever-expanding codebases so I can easily burn tokens. Not to mention, structured agentic coding with adverserial reviews & task organization is not token-efficient. Additionally, for the problems I'm working on, only xhigh or high reasoning gives me worthwhile results while saving time. There are definitely configurations where default consumption doesn't work.
For reference, I used 15 billion tokens (most of it cached) last month on my day job's enterprise plan. That doesn't include my personal plans' usage.
> I personally use my OpenAI subscription pretty heavily, 2-3 agents running practically all day on various tasks but I never even get close to running into limits
Same. But if I was working for an organization that measured token usage, you can bet I would be doing things like creating a cron job that uses claude to create a customized bespoke report update of the current status of all my open assigned tickets and message that to myself 4 times a day... token burn for zero purpose whatsoever.
I make like 2 prompts a week to gemini flash on the weband get more done than all the people that are exhibiting literal manic behavior in the way they use LLMs.
I really wish the management behind these dumb ideas couldnt just quietly pretend they never did it once it goes out of fashion.
The fact that somebody established a leaderboard for tokenmaxxing ought to follow you around like a black cloud for the rest of your career once the collective hallucination lifts and people realize just how monumentally stupid it was.
Alas they do all these stupid things together which makes it seem more defensible and then everybody forgets.
What is the OP talking about. $/unit intelligence is going down rapidly. You can achieve what would have been considered miracles in 2022 with < $10.
Absolutely, though I think the expectations are being set by those who have watched too many "OpenClaw business on autopilot" videos.
Capex and revenue should not be compared like this, unless revenue isn't growing.
I seldom use my PC anymore ever since i got a laptop. with the cost per token increasing along with the random "features" where models will just eat through your tokens in one hour. I really have been tempted to turn my PC into a server to run local models on there
Insofar as I can tell, inference is on a certain path toward becoming "free". The models are now extremely powerful on high-end consumer hardware, and the efficiency trend seems likely to continue.
Here is a recent non-rigorous benchmark I ran against a bunch of models. Qwen3.6 35B A3B fine-tuned with opus data runs plenty fast on my local machine and produce outstanding results - easily in the top 5, comparable to GPT 5.5 Pro (which is $180/mtok).
https://gistpreview.github.io/?31d66ef69e4aed3efae1aec69d86c...
I've predicted for years now that the industry will head down the path of the virus scanning vendors: selling subscriptions to be able to download the latest versions of models. I simply don't see how any other business model is remotely viable, except at the very highest end of inference or video gen.
That local hardware is not consumer though but prosumer. Consumer is a 500$ laptop running that and that is not currently the case.
I get similar results for deepseek and opus but opus is way faster. I guess deepseek streams thinking and makes it slower?
Has this not been true for a long time now? Most companies have had enterprise/business level prices that was highly connected to usage for a what feels like at least a year.
kind of sobering to realize that whether your job can be profitably automated away comes down to what $/token some hyperscale AI provider can deliver… I suppose it’s nice that this article highlights some upward pressure on that number.
Inference costs absolutely did fall. And even more so when looking at intelligence it buys you.
eg compare say gpt 3.5 to latest deepseek. Both cheaper and more at more capable
> Memory for 4x expensive
> Did we collectively forget second order thinking?
I bought 2x 16Gb NVIDIA cards this week because I don’t see hardware getting cheaper anytime soon, and because of that I totally don’t see the point of “waiting until prices go lower for graphics cards” because that might not for a long time yet!
In fact, if you include factoring in world events (and the ones that haven’t happened yet but eventually will e.g. China’s 2027 long planned take of Taiwan), then there’s no way graphics prices are going to be accessible to mere mortals until at least 2028.
But my real reasoning is that you’re going to see a flood of OpenAI and Anthropic users leave because of a) increasing pricing plans, and b) impeding business laws on the horizon about protecting sovereign data from AI (i.e data in cloud for training is a no no).
So what happens when people and companies one by one start leaving the SOTA AI cloud for from-good-enough-to-wow models? RAM and graphics cards become the new toilet paper, which is going to double again current prices.
Upgrade now before it’s too late folks!
Some of these coming price increases will move dev work back to dedicated shops and teams when individuals and non-devs won't want to pay the AI bill to finish and ship their projects.
An outside small dev shop or internal dev team can pay these prices and spread the cost over several customers or departments, but the era of giving everyone AI and telling them to dev stuff is about to be over.
This is only true if your world is limited to openai, antropic and alike.
There are a whole bunch of companies somewhere else in the world that are getting better and cheaper every month, hardware side included. all without the infinite VC money
It's hard to take this piece seriously if he's citing _Ed Zitron's_ math, and equally hard to make the blanket statement that flat-rate plans = "the current AI pricing". But yes, those pricing models were pretty silly and unsustainable.
Get back to me when there's an AI company that's actually profitable and we can compare their service and pricing.
Claiming that there's some small subset of their services (like inference per token) that's "profitable" doesn't mean anything when it relies on everything else that company is still paying for. If you could make money from it at current prices - why aren't they?
Otherwise it's just "how much they're willing to subsidize".
There is probably going to be a quarter or two of profits when the prices dramatically increase. Vibe coding techbros are hooked on the Iron Lung and may not want to get off.
At my work are multiple developers bragging about overnight AI usage to solve problems hands off. Yes they are wasting money and resources but the fad is here. People be vibe coding for now.
In like 6 months when all the costs need to be paid and the prices go up, we will see if these companies stay profitable. But I'm of the opinion that the vibe coding tech bros are more than enough to sustain a short or even medium term profit for these companies. Just on fad-energy alone (see OpenClaw)
------------
I dunno where this is all going. But I do have faith in human ingenuity still. Things are changing, possibly for the worse, but we need to make the best of it.
The worst of behaviors is wasteful and blatant fraud. There's something useful here though.
You are comparing two different model. It's like saying roadster is more expensive than model S. No model pricing actually increased, and I am using GPT-4o in the same price as it was before.
You can see price vs performance in artificial analysis and the the pareto optimal is all just 6 months old model.
This is slightly more tasteful slop than average (I'm thinking probably Claude rather than ChatGPT?), but it's still 100% AI written: https://www.pangram.com/history/c55ab69b-e0a9-49a0-8056-2fcd...
This... is not a reliable AI detection method at all.
You are incorrect. There, now we've both made unsupported assertions. Care to provide any evidence for your position?
For what it's worth, when I provide a Pangram link it's because I can already tell something is AI and I'm attempting to provide objective third-party confirmation so the conversation doesn't just degrade into me asserting that I have superior taste to you.
Pangram is highly reliable.