This is the problem with cloud models, you build a "predictable" workflow then they remove it with a new and improved one that is less deterministic and often costs more. If you use a local model discontinuation is no longer a thing to worry about.
This is a huge (obvious) risk I've seen very little talk/concern on.
It's not even the new one being "less deterministic and often costs more".
Application prompts overfit the model they are using to get the output they want. Switch model, the prompt no longer produces the output you expect and you may not be able to get the output you need anymore.
If a model is critical for your application/business you want the ability to run it yourself otherwise you are stuck when it becomes unavailable.
I am more concerned about the cost step up from Gemini 2.5 Flash to 3.5 Flash, with the latter being roughly 3x more expensive. I thought the intention of the Flash models was to be relatively low-latency and more affordable compared to Pro, but the newer Flash models aren’t being priced as such. Then again, the era of cheap and plentiful AI might be coming to an end…
Yeah IIRC the latest Pro is $12 and Flash is $9 which is not the usual 2X-3X multiplier we see separate model grades. It also puts Flash now about 2X GLM 5.2, which is a highly capable open weight model.
I think the thing is that 3.5 flash is actually similarly capable on a lot of tasks that matter and is faster. Pro is more specialised in the direction of mathematical reasoning and stuff.
I feel this way about gpt-5-nano (EOL December 2026). It seems like the open weight models have progressed a long way since these old models were released though. Deepseek V4 Flash is even cheaper than gpt-5-nano. I'm still going to pay a cloud provider to run it for me, I'm not local inference pilled yet, but I _can_ run it myself in the future if worse comes to worst.
Objectively testable evals are one thing, but how does one judge whether a new model is adequately reproducing the subjective "writing style" of an old model that you've gotten accustomed to the feel of?
I love how there is a "Please do not discontinue gemini-2.0-flash[-lite], 2.5 is NOT an equivalent" from Feb 20th. Getting too attached to models is a smell.
It's not a smell. Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it.
If they were on open weights, at some point the provider would deprecate it, probably with worse notice
And self-hosting would probably have been more expensive unless they had massive volume: Deepseek V3.x would have been the comparable open weight model for the performance and isn't that cost effective until hosted across multiple nodes with large batch sizes
This is how development looks like for many years now, constant rewrite on the horizon. I think LLM development hype surpassed Blockchain and JS frameworks craze of decade ago.
Its almost a given considering how fast this field moves. Also, what kind of workflow structure would someone have that a single specific model is the only one that would perform acceptably?
Programs are tightly shaped to their dependencies and software frequently breaks when bumping a dependency versions. That's why we have package managers that are carefully built around lock files. Models should be thought of like a dependency, not just an endpoint to a workflow.
In the post the issue is performance. Are you saying that getting too attached to performance is a smell? That sounds very odd.
It's not because a model performs better in some applications (often by fine-tuning to get better scores at specific tests) that it is better across the board or that we have to believe the company releasing the model with a high number 3 > 2 so that it is commonly accepted as better.
Pushing the reasonnning further: f you need an Opus level performance then not accepting GPT 3 isn't a smell.
The smell is about folks being attached to the same model because they're used to it. Just like any UI change in an app that one uses everyday is annoying.
What's often missing is the actual engineering. If you have evals for your use case, you use these to adjust to the new model. DSPy has great prompt engineering constructs it ships with.
The smell is all about vibing, where a model feels better because the structure of the answers is more familiar to a person, instead of engineering where given constraints and input/outputs one is using/bending the system to fulfil the requirements.
the 1.5 and 2.0 flash models were absolute beasts. They were very cheap, and _very_ fast. We contemplated moving some of our fine tuned workloads to them because we would have gotten very substantial total latency reductions for our workloads.
However, they are aggressively deprecating them (OpenAI is as well), and replacing with newer models. These newer models are all reasoning models, and importantly, only bear the flash name. They are not fast. And they are very expensive!
We have benchmarks for our use cases, and every generation after Gemini 2.0 Flash has been a grim hit on price/performance. Costs have gone up, throughput has gone down, and performance has improved very slightly (and regressed on a few things).
I built some BigQuery workflows on 2.0 and 2.5 flash lite that are something like 6x more expensive with 3.1 flash lite.
I tried 3 flash for months and it didn’t work using Googles own vertexai integration because it’s been in preview mode for months.
Not wanting to pay significantly more and do a bunch of rework isn’t a smell.
They left a large gap in their new pricing vs the prior generation, and if you had a working use case that sucks. The model is >99% reliable for my use case so there’s nothing to gain from a smarter model.
Agree with the observation others have made. The only true solve if a specific model version is critical to your application or workflow, you need to host the model yourself so you have control over it. You don't want to be stuck getting rug-pulled by a model provider.
And as another commenter pointed out - in particular for Google of all companies - expect that the rug pull can and will happen. They're not known for keep anything around for very long.
It's such a good model for the price, for a lot of tasks it outperforms gpt5 at 3x the speed and 1/5 the price. The price jump from 2.5->3->3.5 has been so high.
some of the tool calling is better, its better at knowing how to use a sequence of tools in a real world scenario. things like glm 5.2 will spam tool calls like 100 times. gemini model will just use the tools as you would expect
im always convinced people with takes on the open source models have never actually used them in a production agentic system
Qwen 35ba3b is ok, but it uses like 3x as many tokens as gemini 2.5 flash so you end up paying about the same to do an agentic run with gemini 2.5 flash and qwen 35ba3b.
in this case, the other frontier shops are all doing the same thing, so maybe the advice here should be "don't use closed weight models in production"?
Google has a history of pulling the rug on their paying customers and offering 0 support when it is convenient for them. You have no recourse. There are a billion cloud providers to choose from, this is not the first time this has been on the first page of HN.
Where do people get ideas like this? In what world does this make sense?
You have several choices:
1. Work with a supplier and sign a contract guaranteeing support for whatever period of time you want at a mutually agreeable price
2. Host your own stack to depend on and support it for however long you want
3. Accept that you're paying for a service and that it can go away at any time.
Companies aren't obligated to support things forever and they aren't obligated to open them up when they no longer feel it's worth supporting them. Claiming they should is absurd.
>they aren't obligated to open them up when they no longer feel it's worth supporting them.
Creating a legal obligation to release the weights of discontinued models doesn’t seem absurd. These models are built on existing publicly available information; a requirement that it be returned to the commons once it is no longer in commercial use hardly seems like a substantial regulatory burden.
There are plenty of open weights models available already. If the ability to keep running the same model is important to you, then choose one of those.
Presumably, a “Stop Killing AI” movement, mirroring the Stop Killing Games movement, would require a provider that revokes access a previously available model to make it open weights at the time of death.
On the surface, there appears a difference between buying a game and paying for llm processing time. You haven't bought the model, so it is unclear to me why the same argument ought to hold up.
UGH why are they killing this model? This is one of the best models you can use in an API for a large swath of tasks. It's kind of the perfect trifecta of fast, cheap, and smart enough.
I was going to reply that Anthropic, which supposedly is the most capacity constrained of the leading AI labs, still provides access to models as old as Opus 3.
I feel the same way about qwen-2.5-coder. Work yanked it from our internally-availbale models one day, breaking a couple tools that depended on it heavily. I haven't found another model that performs as well for the specific tasks I was doing with it. Like yeah, I could throw some gargantuan model at it but then it would take eons to get the same result that used to take 3 seconds.
I've settled on deepseek-v4-flash as a replacement. Results are just as good, but it's slower.
> clearly benchmark and optimise for a specific model over millions of datapoints > new model comes out > get to do it all over again. At this point just become Cursor and get paid for it.
Interestingly, I found the original nano banana also has the best latency/quality trade-off that new versions can't beat. This might be domain/prompt specific though. I wonder if there is some truth in the saying that something is either new or improved by never "new and improved".
Yes, Gemini 2.5 Flash is well balanced model that meets sweet spot of price vs performance trade-off which is good enough for non-reasoning tasks and offer at competitive price.
I was thinking that somebody (Me? No... Somebody.) should start a foundation to start making archive copies of models... not for usage, but because every model is a (somewhat flawed) snapshot of the state of global knowledge as of the time it was trained. In fifty years, you'll be able to talk to Gemini 2.5 flash and get answers from the perspective of 2025. It might be a valuable for historic and sociological research.
there was a community of people who used (famously sycophantic) gpt 4o as—for lack of a better word—a friend, who were devastated when it was shut down
I suppose at least in this case the loss is not an emotional one?
There will be such a massive shift to Qwen VL when Google shoots itself in the foot retiring Gemini 2.5 Flash just because a $1 million/yr L7 wanted to show initiative to become a $1.2 million/yr L8
Is it really? A 9B model is equivalent? Honest question, as I haven't spent that much time with the 9B variant or Flash 2.5. But that seems like a pretty bold claim for such a small model. I assumed Flash 2.5 was considerably larger, but maybe I'm wrong?
Second this, they keep getting performance upgrades too. Z Lab had been publishing dflash addons and boosting their tgen 2-3x. I'm looking at doing comparative evals right now
So you’re telling me, these people have workflows thats so tightly integrated to gemini-2.5-flash that no other model matches it’s performance? Really?
Have they really looked at all alternatives and found none to be a viable option?
I might have underestimated how good 2.5-flash was. I understand the issue with pricing though.
This is why I believe, for a company, to never be reliant on closed-weight models.
They appear to be trying lock-in, or some sort of way to make Gemini family the only logical choice on their cloud. They don't offer the most desired open weight models per-token, so we found another vendor and are less likely to use Google services going forward (for more reasons than this)
This is the problem with cloud models, you build a "predictable" workflow then they remove it with a new and improved one that is less deterministic and often costs more. If you use a local model discontinuation is no longer a thing to worry about.
This is a huge (obvious) risk I've seen very little talk/concern on.
It's not even the new one being "less deterministic and often costs more".
Application prompts overfit the model they are using to get the output they want. Switch model, the prompt no longer produces the output you expect and you may not be able to get the output you need anymore.
If a model is critical for your application/business you want the ability to run it yourself otherwise you are stuck when it becomes unavailable.
I am more concerned about the cost step up from Gemini 2.5 Flash to 3.5 Flash, with the latter being roughly 3x more expensive. I thought the intention of the Flash models was to be relatively low-latency and more affordable compared to Pro, but the newer Flash models aren’t being priced as such. Then again, the era of cheap and plentiful AI might be coming to an end…
Yeah IIRC the latest Pro is $12 and Flash is $9 which is not the usual 2X-3X multiplier we see separate model grades. It also puts Flash now about 2X GLM 5.2, which is a highly capable open weight model.
I think the thing is that 3.5 flash is actually similarly capable on a lot of tasks that matter and is faster. Pro is more specialised in the direction of mathematical reasoning and stuff.
I use flash 3.5 a lot. Its significantly worse than gpt 5.5.
I feel this way about gpt-5-nano (EOL December 2026). It seems like the open weight models have progressed a long way since these old models were released though. Deepseek V4 Flash is even cheaper than gpt-5-nano. I'm still going to pay a cloud provider to run it for me, I'm not local inference pilled yet, but I _can_ run it myself in the future if worse comes to worst.
Objectively testable evals are one thing, but how does one judge whether a new model is adequately reproducing the subjective "writing style" of an old model that you've gotten accustomed to the feel of?
I love how there is a "Please do not discontinue gemini-2.0-flash[-lite], 2.5 is NOT an equivalent" from Feb 20th. Getting too attached to models is a smell.
It's not a smell. Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it.
> Why should these developers rebuild a core piece of their stack every few months?
That's what they signed up for when established a hard dependency on an subscription online-only LLM model.
If they were on open weights, at some point the provider would deprecate it, probably with worse notice
And self-hosting would probably have been more expensive unless they had massive volume: Deepseek V3.x would have been the comparable open weight model for the performance and isn't that cost effective until hosted across multiple nodes with large batch sizes
Sounds like something you'd need to evaluate before adopting then.
This is how development looks like for many years now, constant rewrite on the horizon. I think LLM development hype surpassed Blockchain and JS frameworks craze of decade ago.
Its almost a given considering how fast this field moves. Also, what kind of workflow structure would someone have that a single specific model is the only one that would perform acceptably?
Programs are tightly shaped to their dependencies and software frequently breaks when bumping a dependency versions. That's why we have package managers that are carefully built around lock files. Models should be thought of like a dependency, not just an endpoint to a workflow.
It's kind of the same problem with cloud in general (though that moves much slower).
If you want to be sure to be in control, then host it yourself
In the post the issue is performance. Are you saying that getting too attached to performance is a smell? That sounds very odd.
It's not because a model performs better in some applications (often by fine-tuning to get better scores at specific tests) that it is better across the board or that we have to believe the company releasing the model with a high number 3 > 2 so that it is commonly accepted as better.
Pushing the reasonnning further: f you need an Opus level performance then not accepting GPT 3 isn't a smell.
The smell is about folks being attached to the same model because they're used to it. Just like any UI change in an app that one uses everyday is annoying.
What's often missing is the actual engineering. If you have evals for your use case, you use these to adjust to the new model. DSPy has great prompt engineering constructs it ships with.
The smell is all about vibing, where a model feels better because the structure of the answers is more familiar to a person, instead of engineering where given constraints and input/outputs one is using/bending the system to fulfil the requirements.
the 1.5 and 2.0 flash models were absolute beasts. They were very cheap, and _very_ fast. We contemplated moving some of our fine tuned workloads to them because we would have gotten very substantial total latency reductions for our workloads.
However, they are aggressively deprecating them (OpenAI is as well), and replacing with newer models. These newer models are all reasoning models, and importantly, only bear the flash name. They are not fast. And they are very expensive!
I test workloads with multiple closed and at least one open model now. Good to have a backup on 503s or credits run out.
have you experimented at all with the deepseek flash models?
We have benchmarks for our use cases, and every generation after Gemini 2.0 Flash has been a grim hit on price/performance. Costs have gone up, throughput has gone down, and performance has improved very slightly (and regressed on a few things).
That's like saying 'getting attached to locked dependencies for your app is a smell'.
But this could be framed as 'getting attached to an API revision when a new one is available'...
I can see it both ways, tbh.
I built some BigQuery workflows on 2.0 and 2.5 flash lite that are something like 6x more expensive with 3.1 flash lite.
I tried 3 flash for months and it didn’t work using Googles own vertexai integration because it’s been in preview mode for months.
Not wanting to pay significantly more and do a bunch of rework isn’t a smell.
They left a large gap in their new pricing vs the prior generation, and if you had a working use case that sucks. The model is >99% reliable for my use case so there’s nothing to gain from a smarter model.
If they don't want to host it maybe they could open source it. This would probably be a win-win situation.
How is that a win for them?
Agree with the observation others have made. The only true solve if a specific model version is critical to your application or workflow, you need to host the model yourself so you have control over it. You don't want to be stuck getting rug-pulled by a model provider.
And as another commenter pointed out - in particular for Google of all companies - expect that the rug pull can and will happen. They're not known for keep anything around for very long.
It's such a good model for the price, for a lot of tasks it outperforms gpt5 at 3x the speed and 1/5 the price. The price jump from 2.5->3->3.5 has been so high.
i have found google models outperforming other models in actual agentic workflows
I find that Gemini flash 2.5 performs about as well as Claude sonnet for non coding agentic flows except it’s actually fast enough
some of the tool calling is better, its better at knowing how to use a sequence of tools in a real world scenario. things like glm 5.2 will spam tool calls like 100 times. gemini model will just use the tools as you would expect
im always convinced people with takes on the open source models have never actually used them in a production agentic system
Qwen 35ba3b is ok, but it uses like 3x as many tokens as gemini 2.5 flash so you end up paying about the same to do an agentic run with gemini 2.5 flash and qwen 35ba3b.
I don't know how many times people will need to learn this: Do not use Google in Production.
in this case, the other frontier shops are all doing the same thing, so maybe the advice here should be "don't use closed weight models in production"?
Google has a history of pulling the rug on their paying customers and offering 0 support when it is convenient for them. You have no recourse. There are a billion cloud providers to choose from, this is not the first time this has been on the first page of HN.
I'm aware. regardless, when it comes to models, the advice to "not use Google in Production" falls short.
It ain't just google.
Why not a "stop killing AI" movement?
If a company deploys a paid AI model and makes people depend on it, they need to dump the weights at EOL.
Where do people get ideas like this? In what world does this make sense?
You have several choices:
1. Work with a supplier and sign a contract guaranteeing support for whatever period of time you want at a mutually agreeable price
2. Host your own stack to depend on and support it for however long you want
3. Accept that you're paying for a service and that it can go away at any time.
Companies aren't obligated to support things forever and they aren't obligated to open them up when they no longer feel it's worth supporting them. Claiming they should is absurd.
>they aren't obligated to open them up when they no longer feel it's worth supporting them.
Creating a legal obligation to release the weights of discontinued models doesn’t seem absurd. These models are built on existing publicly available information; a requirement that it be returned to the commons once it is no longer in commercial use hardly seems like a substantial regulatory burden.
I mean, claiming the have a moral imperative to do it might be a bit of a stretch, but it sure would be nice - can’t blame people for wanting things.
It's not a legal obligation, no, but neither should you as a customer accept a vendor that treats you like that.
You're not going to find many people to do business with in the world if that's your bar.
> vendor that treats you like that
You don't have to use Big Ai offerings, there are other options. Between deprecation and uncle sam, dependency/business risk appears to be increasing.
It's a calculation and choice that comes with consequences any way you land.
If you paid a one-time fee for an offline model, and then you were revoked access to it, that would apply.
If you are paying an ongoing subscription for a service, I'd advise you not to rely on it too much or keep a list of alternatives.
There are plenty of open weights models available already. If the ability to keep running the same model is important to you, then choose one of those.
Presumably, a “Stop Killing AI” movement, mirroring the Stop Killing Games movement, would require a provider that revokes access a previously available model to make it open weights at the time of death.
These are not analogous. If you paid for an offline model, and you were somehow revoked access because it was phoning home, that would be closer.
If you were paying for an ongoing subscription for a service, that would be something different.
On the surface, there appears a difference between buying a game and paying for llm processing time. You haven't bought the model, so it is unclear to me why the same argument ought to hold up.
UGH why are they killing this model? This is one of the best models you can use in an API for a large swath of tasks. It's kind of the perfect trifecta of fast, cheap, and smart enough.
Why does Google constantly kill off good things?
because they keep these models loaded, and they can't just arbitrarily load up whatever models you want.
but it's more likely just a business case: they need you buying higher tier model output. They know whose doing what, so someone needs their 3Q bonus.
I was going to reply that Anthropic, which supposedly is the most capacity constrained of the leading AI labs, still provides access to models as old as Opus 3.
But then I realized Opus 3 is an outlier, and Anthropic has removed access to relatively more recent models. https://platform.claude.com/docs/en/about-claude/model-depre...
I wonder what the deal is with Opus 3.
I believe a lot of people prefer it for "creative" writing.
I feel the same way about qwen-2.5-coder. Work yanked it from our internally-availbale models one day, breaking a couple tools that depended on it heavily. I haven't found another model that performs as well for the specific tasks I was doing with it. Like yeah, I could throw some gargantuan model at it but then it would take eons to get the same result that used to take 3 seconds.
I've settled on deepseek-v4-flash as a replacement. Results are just as good, but it's slower.
the writing is on the wall for it, I have switched to gemma-4-26b-a4b.
At least in benchmarks, it scores higher and is faster.
> clearly benchmark and optimise for a specific model over millions of datapoints > new model comes out > get to do it all over again. At this point just become Cursor and get paid for it.
Interestingly, I found the original nano banana also has the best latency/quality trade-off that new versions can't beat. This might be domain/prompt specific though. I wonder if there is some truth in the saying that something is either new or improved by never "new and improved".
Yes, Gemini 2.5 Flash is well balanced model that meets sweet spot of price vs performance trade-off which is good enough for non-reasoning tasks and offer at competitive price.
I wonder if some day we might see Archive.org organizations preserving older models as operating costs go down
I was thinking that somebody (Me? No... Somebody.) should start a foundation to start making archive copies of models... not for usage, but because every model is a (somewhat flawed) snapshot of the state of global knowledge as of the time it was trained. In fifty years, you'll be able to talk to Gemini 2.5 flash and get answers from the perspective of 2025. It might be a valuable for historic and sociological research.
Maybe there's also a security aspect to this, older models are probably worse against prompt injection, etc.
I really like Gemini 2.5 Flash Lite because it's a dirt cheap model that support every input modalities.
At least now MiMo v2.5 exists and can be used as another dirt cheap multimodal model.
there was a community of people who used (famously sycophantic) gpt 4o as—for lack of a better word—a friend, who were devastated when it was shut down
I suppose at least in this case the loss is not an emotional one?
Yes, these people did not like 4o going away.
This is one of the most dystopian subreddits:
https://www.reddit.com/r/MyBoyfriendIsAI/
There will be such a massive shift to Qwen VL when Google shoots itself in the foot retiring Gemini 2.5 Flash just because a $1 million/yr L7 wanted to show initiative to become a $1.2 million/yr L8
Can't run Qwen 3.6 35B A3B? Even Qwen 3.5 9B is comparable.
Is it really? A 9B model is equivalent? Honest question, as I haven't spent that much time with the 9B variant or Flash 2.5. But that seems like a pretty bold claim for such a small model. I assumed Flash 2.5 was considerably larger, but maybe I'm wrong?
Pretty much. It even beats it in a few benchmarks: https://artificialanalysis.ai/models/comparisons/qwen3-5-9b-...
Qwen 3.6 and Gemma 4 small models are in a league of their own.
Second this, they keep getting performance upgrades too. Z Lab had been publishing dflash addons and boosting their tgen 2-3x. I'm looking at doing comparative evals right now
https://huggingface.co/collections/z-lab/dflash
So you’re telling me, these people have workflows thats so tightly integrated to gemini-2.5-flash that no other model matches it’s performance? Really?
Have they really looked at all alternatives and found none to be a viable option?
I might have underestimated how good 2.5-flash was. I understand the issue with pricing though.
This is why I believe, for a company, to never be reliant on closed-weight models.
gemini 2 and 2.5 were great models for quick-and-dirty OCR
It was fine to lose 2, but 2.5 will be dearly missed as it hit the sweet spot in terms of cost-performance :/
sucks we use 2.5 flash/lite in our company it handles millions of requests a day
theres nothing in its price range that provides the same all around perf
as noted, gemini 3 flash is expensive
really not liking google these days they are not hungry anymore
They appear to be trying lock-in, or some sort of way to make Gemini family the only logical choice on their cloud. They don't offer the most desired open weight models per-token, so we found another vendor and are less likely to use Google services going forward (for more reasons than this)
"Don't discontinue Google RSS reader!"
How about you stop relying on Google products? You've learned nothing after all these years?
Isn't asking Google to not discontinue a product a bit like asking the tide to not rise?
That, and while you’ve chained yourself to the floor of the littoral zone previously.
And worse, they didn't say please.
"Do not get rid of GPT 4o"