Category narrowing to this degree really has diluted the term "SOTA" quite a lot
You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!
Irrelevant. A term can be well-defined and used correctly while also being diluted. In this case, I am pointing out that it's diluted by narrowing the category in which the claim is being made. I have no doubt that in this narrowed category, the term is being used correctly. This has no bearing on the fact that by narrowing the category in that way, this claim is made remarkably less impressive or interesting to many people hearing it, including myself
What's funnier is that after cherry picking models for comparison, they further cherry picked Inking for size comparison instead of better and smaller Beam 1.0
The bar chart also shows Nemotron 3 Ultra instead of smaller and better Nemotron 3.5 Super.
US Gov won't let Chinese models be hosted on any internal infra, not even supposed to use them for local AI. So there is absolutely a market for US-made or non-Chinese models there.
It's great to see another competitive open weight model. Congratulations!
But as is, it's probably hard to become popular amongst hobbyists and local model enthusiasts like Qwen's 27B class models or MOE models because it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware, and seems like the goal/mission of the company behind this model is to automate chip design.
So if I have to assume and connect the dots, this is for the companies who want to use AI in air-gapped environments for chip design so that they protect their IPs (not leaking to third party AI labs/hostile parties) and can afford lots of server grade inference hardware for a quick unlock with the big dense model.
Category narrowing to this degree really has diluted the term "SOTA" quite a lot
You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!
The US bit got me as well.
I guess it makes sense - there is a segmentation where users may want to run on domestic "frontier LLMs."
No, the term is well defined. It means being on the Pareto curve of performance vs size, such as: https://tinyurl.com/26sw35fw
Irrelevant. A term can be well-defined and used correctly while also being diluted. In this case, I am pointing out that it's diluted by narrowing the category in which the claim is being made. I have no doubt that in this narrowed category, the term is being used correctly. This has no bearing on the fact that by narrowing the category in that way, this claim is made remarkably less impressive or interesting to many people hearing it, including myself
well yes but the y axis heavily depends on the benchmark you choose
Congratulations I guess but it's a pretty bad look to not even mention those other open weight models.
What's funnier is that after cherry picking models for comparison, they further cherry picked Inking for size comparison instead of better and smaller Beam 1.0
The bar chart also shows Nemotron 3 Ultra instead of smaller and better Nemotron 3.5 Super.
We've always been at war with eurasia.
Seriously though, it's just easier to pretend that China doesn't exist when they're stomping us this badly.
Do those other models usually mention all the competition? I haven't noticed
Right? These are not likely to be meaningful points of comparison to anyone but the authors.
Maybe Palantir will use it or something. There probably is a market for US models but I don't know how large it will be.
US Gov won't let Chinese models be hosted on any internal infra, not even supposed to use them for local AI. So there is absolutely a market for US-made or non-Chinese models there.
It's great to see another competitive open weight model. Congratulations!
But as is, it's probably hard to become popular amongst hobbyists and local model enthusiasts like Qwen's 27B class models or MOE models because it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware, and seems like the goal/mission of the company behind this model is to automate chip design.
So if I have to assume and connect the dots, this is for the companies who want to use AI in air-gapped environments for chip design so that they protect their IPs (not leaking to third party AI labs/hostile parties) and can afford lots of server grade inference hardware for a quick unlock with the big dense model.
Neat!
This model scores 80.5 on Terminal Bench 2.1
For reference Qwen 3.8 Flash Next (which is bigger than this model but probably cheaper to run due to MoE) scores 84.3, and Qwen 3.8-27B scores 73.0.
I think this model could be competitive, depending on how much it was benchmaxxed compared to Qwen?
When a post doesn’t mention Qwen for comparison, you know the performance isn’t going to be great.
I mean Qwen 3.8 27B hits 73% on TB2.1 and 42.2% on DeepSWE....
So we have a Recursive and a Ricursive, both making LLMs?
Time to start Rickursive, the lab that will never give you up and never let you down.
And it should acquire the existing two
Roll them up, you might say.
we need RichUrsive
There seems to be some recursion going on here.
One making self-improving AI via AI-chip co-design, the other relying only on software for self-improving AI.
I hope they never have to call each other about anything. It could get confusing if neither knows who is the base case.
Not to be confused with https://recursive.com/
Would love a 27B model. Wonder if we'll get a distillation.
The comparison chart is befuddling.
I'm sorry but what is "U.S." implying here?
Is this sponsored by the us gov? did it only train on US data? does it only work on the US?
It's a US group that made it, in the same vein that OpenAI and Anthropic have US models.
It means that it's an American AI lab behind it.
Only benchmarking against other garbage american models?
LMFAO get out of my feed