> "the charts below compare it with GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8, and Claude-Opus-5"
There should be a rule against releasing seemingly-obsolete comparisons without explaining why you're doing it. If you don't give a cost reason, it makes you look like you're trying to hide something.
I'm not saying that the given comparison is inherently bad. I'm saying that they failed to include any useful-to-potential-users reason for why those specific non-current model versions are meaningful.
I would suspect the point of the older comparisons is to be able to show that local models have caught up to where frontier models were a few months ago.
I know this is besides the point but I genuinely remember a time not very long ago that people used to say that they would kill for an claude opus 5 model at home or an open weights model and that opus 4.8/5 are the first models which finally get what they are suggesting.
Seems like our expectations and the floor has been rising at such an exponential rate.
Has someone created a benchmark for our expectations to see how much they've grown for new model releases? I'd be curious to see some of the data :-D
> a benchmark for our expectations to see how much they've grown for new model releases?
Posters have been saying they'd be happy with open-weight Fable 5 since before F5 was public and I saw one poster on HN say it last week. And with those posts there's usually a few mentioning the truism 'goalposts tend to move' and we haven't met our future selves or seen the future context we'll be in.
Studies show humans are poor predictors of our own future wants, needs, preferences and behaviors and few are fully aware how far and often we gradually move goalposts. Some of those posters weren't really making a precise, quantified statement. Many were expressing something more directional like "Wouldn't it great if {best, most costly thing I want but can't get or afford} were {readily available free or cheap } ?" Those variables update every time the sentence is output.
And in recent months models are saturating more of the benchmarks HN posters relate to and more of the real-world workloads we have. Six month ago, model flaws were more obvious and common. They still happen today, but for more of us, they're rarer, more complex and workload dependent. It's getting harder or takes longer for us to see clear, consistent differences between similar models and model versions. You can even see it in Simon's fanciful bike-riding pelican 'not-a-benchmark'.
I'm not saying that models are now great at everything (or even close), just that for a lot of us they're good enough at our current things that it's getting harder to personally gauge by how much. So we have to rely on complex composite benchmarks (which increasingly don't reflect any one thing at high granularity) or proxy reports about 'other people's things' like frontier theoretical mathematics.
I know that but given that it's most likely seems to be an open-weights model or that you can still run theoretically at home without being at the whims of Anthropic/OpenAI is still quite extraordinary.
It is a useful comparison since it more precisely defines the model capabilities. The alternative would be 'quite a ways from Astra' which is less helpful. So I don't think they hide anything, they don't assume people don't know about newer models or won't notice.
Seemingly has links to GitHub and HuggingFace, but no actually explanation about what we're supposed to be able to find there "soon". Is this supposed to be downloadable weights? Open weights? Who knows?
One link to "https://longcat.ai/chat/" is provided, which eventually redirects you to https://passport.mykeeta.com/pc/login, did the UI get hacked or something? I went to mykeeta.com which doesn't respond, passport.mykeeta.com serves OpenResty's default page, not sure what's going on but looks strange.
Asking "Hello?" in the chat you get "你好！我是 LongCat，美团研发的大模型助手。有什么可以帮你的吗？" back, which is an explanation what LongCat is in Mandarin. If you then write "What?", it does web-searches trying to explain what "What?" means, in Mandarin.
I'm not really sure this is ready to even be played around with yet. Some more basic prompts seems to tell me that despite the benchmark results, it's fairly dumb, so gut tells me this is benchmaxxed then pushed out as quickly as possible.
When the last LongCat model was released, they sold a large token pack for $2 (or something equally cheap). I used it in a few projects to experiment, it was decent enough for basic grunt work.
For some reason they then decided to force you to login to the API console via the Meituan app, which is when I gave up completely. I think Keeta is an international brand for Meituan, which is probably why they're redirecting you to mykeeta.com to login.
But yeah, from a user-friendliness perspective they're awful.
it started typing backward on firefox lol (resetting cursor to the beginning after every typed character)
after i managed to actually type Hello it did 25 web searches and responded in likely to be Mandarin, I didn't care checking. Likely their web harness just sucks.
For API, this seems competitive based on the released benchmarks. For running at home, this is pointless when DS4.1 is more or less equivalent but half the total parameters and less than half the active parameters.
2 turns requesting information and indicating a preference for web search vs answering from memory and it fell over. Thinking block was a mix of Chinese and English, which is fine, but both questions were in English. Second turn answered in Chinese. When told “I don’t speak Chinese” it started to answer the first turn again, then crashed.
I find the benchmark results simply unfathomable after watching that performance.
The barplots are being miscolored, so only their own model is visible. Its hard to compare them. It highly suggests that the blog post was never looked at by a human.
> "the charts below compare it with GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8, and Claude-Opus-5"
There should be a rule against releasing seemingly-obsolete comparisons without explaining why you're doing it. If you don't give a cost reason, it makes you look like you're trying to hide something.
I'm not saying that the given comparison is inherently bad. I'm saying that they failed to include any useful-to-potential-users reason for why those specific non-current model versions are meaningful.
I would suspect the point of the older comparisons is to be able to show that local models have caught up to where frontier models were a few months ago.
This isn't really a local model though, it's in the same class as Kimi K3.1 which it is not benchmarked against.
I know this is besides the point but I genuinely remember a time not very long ago that people used to say that they would kill for an claude opus 5 model at home or an open weights model and that opus 4.8/5 are the first models which finally get what they are suggesting.
Seems like our expectations and the floor has been rising at such an exponential rate.
Has someone created a benchmark for our expectations to see how much they've grown for new model releases? I'd be curious to see some of the data :-D
> a benchmark for our expectations to see how much they've grown for new model releases?
Posters have been saying they'd be happy with open-weight Fable 5 since before F5 was public and I saw one poster on HN say it last week. And with those posts there's usually a few mentioning the truism 'goalposts tend to move' and we haven't met our future selves or seen the future context we'll be in.
Studies show humans are poor predictors of our own future wants, needs, preferences and behaviors and few are fully aware how far and often we gradually move goalposts. Some of those posters weren't really making a precise, quantified statement. Many were expressing something more directional like "Wouldn't it great if {best, most costly thing I want but can't get or afford} were {readily available free or cheap } ?" Those variables update every time the sentence is output.
And in recent months models are saturating more of the benchmarks HN posters relate to and more of the real-world workloads we have. Six month ago, model flaws were more obvious and common. They still happen today, but for more of us, they're rarer, more complex and workload dependent. It's getting harder or takes longer for us to see clear, consistent differences between similar models and model versions. You can even see it in Simon's fanciful bike-riding pelican 'not-a-benchmark'.
I'm not saying that models are now great at everything (or even close), just that for a lot of us they're good enough at our current things that it's getting harder to personally gauge by how much. So we have to rely on complex composite benchmarks (which increasingly don't reflect any one thing at high granularity) or proxy reports about 'other people's things' like frontier theoretical mathematics.
> I genuinely remember a time not very long ago that people used to say that they would kill for an claude opus 5 model at home
I don't think you're going to be running this 1.6 trillion parameter model at home.
I know that but given that it's most likely seems to be an open-weights model or that you can still run theoretically at home without being at the whims of Anthropic/OpenAI is still quite extraordinary.
It is a useful comparison since it more precisely defines the model capabilities. The alternative would be 'quite a ways from Astra' which is less helpful. So I don't think they hide anything, they don't assume people don't know about newer models or won't notice.
Personally I think it's more accurate and useful to compare against competitive models rather than show "X% below the top-of-the-top today"
Better to have people suspect your model isn't as good as you're claiming, than to prove it outright?
It's much cheaper than frontier models, too, so it's not an exact comparison.
The LongCat 2.5 preview came out on 2026-09-25 so I assume it was a relevant comparison at the time?
But this was published yesterday, so they've had time.
Seemingly has links to GitHub and HuggingFace, but no actually explanation about what we're supposed to be able to find there "soon". Is this supposed to be downloadable weights? Open weights? Who knows?
One link to "https://longcat.ai/chat/" is provided, which eventually redirects you to https://passport.mykeeta.com/pc/login, did the UI get hacked or something? I went to mykeeta.com which doesn't respond, passport.mykeeta.com serves OpenResty's default page, not sure what's going on but looks strange.
Asking "Hello?" in the chat you get "你好！我是 LongCat，美团研发的大模型助手。有什么可以帮你的吗？" back, which is an explanation what LongCat is in Mandarin. If you then write "What?", it does web-searches trying to explain what "What?" means, in Mandarin.
I'm not really sure this is ready to even be played around with yet. Some more basic prompts seems to tell me that despite the benchmark results, it's fairly dumb, so gut tells me this is benchmaxxed then pushed out as quickly as possible.
When the last LongCat model was released, they sold a large token pack for $2 (or something equally cheap). I used it in a few projects to experiment, it was decent enough for basic grunt work.
For some reason they then decided to force you to login to the API console via the Meituan app, which is when I gave up completely. I think Keeta is an international brand for Meituan, which is probably why they're redirecting you to mykeeta.com to login.
But yeah, from a user-friendliness perspective they're awful.
The preview version was free on OpenCode for a couple of weeks, so it certainly works. I didn't feed it anything beyond simple coding tasks though
it started typing backward on firefox lol (resetting cursor to the beginning after every typed character)
after i managed to actually type Hello it did 25 web searches and responded in likely to be Mandarin, I didn't care checking. Likely their web harness just sucks.
For API, this seems competitive based on the released benchmarks. For running at home, this is pointless when DS4.1 is more or less equivalent but half the total parameters and less than half the active parameters.
Same price as DS4.1F: https://longcat.ai/platform/docs/pricing/longcat-2.5
On benchmarks they seem comparable. The wild cards are the cache hit rate and speed.
In my first test, it filled 200k of its one-million-token window in 5 minutes. Right now at least, it seems to be fast.
Update: a second thread went down to a more reasonable ~80 tps.
Wild cat AI lab reaches frontier lab parity with models from 4 months ago.
Surprisingly this is both mundane news for frontier AI lab customers and terrible news for frontier AI labs.
2 turns requesting information and indicating a preference for web search vs answering from memory and it fell over. Thinking block was a mix of Chinese and English, which is fine, but both questions were in English. Second turn answered in Chinese. When told “I don’t speak Chinese” it started to answer the first turn again, then crashed.
I find the benchmark results simply unfathomable after watching that performance.
The barplots are being miscolored, so only their own model is visible. Its hard to compare them. It highly suggests that the blog post was never looked at by a human.
A cat that is not a Mistral model now that is confusing.
Pelicans when lol
Here's one in High: https://www.svgviewer.dev/s/UgGqwY8R
IMO pretty damn good, the bike frame is great, the feet connect to the pedals and the wing to the handlebars, and one leg is behind the bike.
[flagged]