(I can't draw a pelican for this one because Alibaba Cloud have flagged my email address and won't let me pay them for access. So I'm waiting for the open weights release, or for the new model to show up on OpenRouter.)
lol the pelican benchmark is basically the only review process I trust at this point. kinda wild that alibaba of all companies is making it hard to give them money tho, you'd think they'd want prominent devs testing their stuff.
openrouter usually picks these up pretty fast, hopefully it shows up there soon. You can also just download the qwen app and do this in the chat interface using their MCP tools for local dev.
> kinda wild that alibaba of all companies is making it hard to give them money tho
Okay so this is entirely off topic but, are there any options to subscribe to these companies without having a credit card? (I am Brazilian if this makes a difference)
It's kind of wild that a Chinese AI lab will require something like.. a credit card with visa or mastercard label, and very little else.
I find it so strange when someone comments something like this.
Nobody thinks this is okay or that Simon should have to play those games. The point of the reply was that while this is stupid, the solution is trivial.
I’m genuinely very curious what the point of comments like this is. I am not joking, I want to understand.
I see examples like it 50 times a week in random places, and nobody walks me through their mental state to help me understand what they thought the point was.
I'm not sure what the situation is here, but on many sites making alt account to avoid bans can result in a lifetime ban. A high profile figure who's been banned from a service advertising that they still have access to it might be a risk.
Also, these companies (or data brokers) likely share ban and alt account lists, ban evading could hurt the reputation of all your accounts.
These are some pragmatic reasons not to evade bans.
Hackernews users are not immune to developing parasocial relationships with strangers on the internet.
The fawning that some users do over others because of their careers is really unbecoming.
I wish the kind of people who did this stuff would realize that there are only two possibilities here -- the person they're saying it to cringes a little bit time they hear it because they're a normal person or they smile a bit inside because they're mildly narcissistic and therefore not worthy of the incessant praise.
I’m a developer from China. So, is this what Hacker News is all about? Whenever a model comes from China, the comments section stops discussing its technical architecture, optimisation points or real-world performance, and instead starts going on about politics, human rights and all that rubbish? To be honest, we Chinese IT professionals possess a genuine geek spirit. That’s why you’re lagging behind in open-source competitions—it’s got nothing to do with politics; you’ve simply lost sight of your original aspirations.
Unfortunately this is a US centric site, for a US based investor, who themselves and probably most commenters as well who are highly paid silicon valley people who have stakes in ai companies on their side of the pond.
Not to mention the political unrest and fear of losing the technological superiority which they once earned but now is trying so hard to hold on to through legally grey monopolistic practices. You dare not outcompete the US in any tech they can sanction.
At some point people who made genuine innovation and wanted to make the world a better and equal place were replaced with capitalists.
Old politicians who barely understand how to use e-mail had a panic attack due to Anthropics marketing about Mythos and being told that Chinese models were "attacking" (distillation) US frontier labs to train theirs.
They pull the only lever they could in an attempt to stop it, but again, they don't understand how any of it works. The export ban lasted a few weeks yet everyone keeps crying about it like the US is shady.
In reality the entire world is 100% dependent on the output of US and China for frontier AI. Anyone outside of those two countries will be loud about either closing off access. I personally think it is inevitable that China and the US stops producing frontier level open models when they become too valuable.
> I personally think it is inevitable that China and the US stops producing frontier level open models when they become too valuable.
This reads like a paradox, though. If the value of a FOSS model comes from pushing the local frontier, then FOSS releases will be even more attractive in a world with highly capable AI services. The underdog researchers will be motivated to train smaller LLMs that close the gap with the most newest architectures and improvements, democratizing those same benefits to local models. There's not really anything that stops them, besides finding an investor.
The US already has top secret AI technology like the NRO's Sentient. The danger is less the architecture, and more the data that it has access to and the technology that it interfaces with. Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
> Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
There will be a generational leap that makes current LLMs obsolete in every way. Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Humans for millennia have used technological advances to make better weapons. AI will be no different, unfortunately. Wars will be fought with drones and artificial intelligence from now on.
> Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Because China is based in a completely different set of values than the USA. They look at the world differently, and have a different understanding of it.
Your assumption that your attitude is pure human nature is deeply grounded in your culture, which (assuming you are from the USA) is deeply grounded in personal competition rather than co-operation. All that rugged individualism. Other cultures are less based around that, and don't see the world as a zero-sum competition for survival.
I'm no expert in Chinese culture, but there are folks from China on this topic who could probably answer this from their perspective.
Rankings have never been something I’ve cared about. Just as with the smear campaigns against China, there is far too much noise in the world. We simply hope that the geeks who are genuinely committed to making the world a better place can unite and focus on getting things done, rather than being brainwashed by ideology and social media.
> committed to making the world a better place can unite and focus on getting things done
The problem is that nobody can agree on what "better" is, so it is difficult to unite.
And in the meantime, people who don't care about "uniting" or "better" just carry on with whatever their goal is(usually just to make as much money as possible), and that sometimes ends up on net making the world a worse place.
This is a legitimate systematic problem that many of us are concerned about solving. It could easily be more important than the technical problems that geeks such as myself would love to be able to focus on.
I assume that this announcement has been prompted by that of Moonshot AI, which has just announced a 2.8T parameter open-weights LLM, Kimi K3, to be published on Huggingface by 27 July.
Now the response of Alibaba is that they will also publish soon a big open weights LLM, the 2.4T parameter Qwen 3.8.
I wonder if Alibaba has always planned to make this big LLM open weights, or they have chosen to do this now, to better compete with Moonshot AI.
In any case, from this competition in LLMs, we win.
It's hard to say what their motivation is. The Chinese firms seem to be working hard to commoditize intelligence which may be the most effective way to debase American frontier labs. And yeah: it also happens to be really good for humanity.
They want to turn LLMs into a commodity, and watch the US AI labs crash and burn.
There will still be plenty of customers who will pay them to host the models and run inference, even if the weights are open and others can offer competing products. (If necessary, the Chinese government can ban use of foreign inference services by Chinese citizens and businesses to give their own companies a domestic monopoly.)
When their models equal or surpass those from the Western AI labs, they can even stop releasing weights for new models, and keep all the inference revenue for themselves.
Meanwhile, they're still manufacturing much of the hardware that everyone in the world needs in order to run datacenters (see also: Spolsky's "commoditize your complement" essay).
Beyond that, it's a soft-power play. As the world keeps looking at the US more and more skeptically as an ally and superpower, Chinese companies releasing weights for competitive models is a way for China to look better and more world-minded.
I feel like there could also be a simpler explanation.
Why does a debian contributor make debian free, why do they work on this thing anyone can use?
Is it because linux and debian hate windows and iOS and want to see american fail?
No, it's because most debian contributors believe software source code, information, should be free, users should be free to modify the code they use, and that they're building a thing they want to share with the world.
Maybe the chinese AI labs believe AI is powerful and useful, are proud of what they're doing, and want to share it as broadly as they can so everyone can use it.
There doesn't have to be any weird "chinese government" or "they hate the west" type vibes, it could just be the same thing as OSS, they're trying to do what they think is best for the world.
There is clearly an anti-China bias here. Show me comments demonstrating the same level of distrust against Google for open-sourcing projects like Tensorflow, Kubernetes, Flutter, Chromium, etc.
The chromium example is wild. There's an extreme distrust and contempt for chromium becoming the defacto browser and therefore Google becoming the defacto gatekeeper of the web.
I'll supply such a comment: Any software open sourced by any for-profit company, including Google, is a calculated move ultimately intended to increase their bottom line, and it's naive to think otherwise.
No, it is not. This is a common misconception that the large companies are very happy about.
1. The century-old Ford decision wasn't about this. It was about him refusing to pay dividends to shareholders he was feuding with. I.e. dominant shareholder using his control to starve minority shareholders of returns.
The court still let him keep spending huge sums on factories and price cuts that didn't clearly maximize profits.
2. Even if the case was about profit maximization (it wasn't), the judgment was a Michigan state decision. It doesn't have force outside of it.
3. US business law is in practice actually the opposite. A business can do pretty much whatever it wants. This is fundamental.
4. Even if it were illegal (it's not), anything can reasonably be framed as being in the long-term interest of the company, including donating to causes, raising employee wages and so on. Courts never question this.
Just imagine this was a thing. It's completely untenable for this reason. Who is a court to judge that something isn't in the longterm interest of shareholders unless it's literally spending all the company money on yachts for personal use?
5. No company has ever been prosecuted for this, obviously, because it's not a thing that exists. The closest you can get is that there's a potential duty to seek the best price if a company must be sold or broken up. But that's a very specific situation.
It's 100% a myth. Feel free to copy this and spread it when you see someone saying this.
Scroll the front page. Find literally any story that has to do with a major US tech company. Open the comment section. Look at the the top comment. It will be negative. Most of the other top comments as well. Trying to gaslight us into not believing our own eyes ..
"Drilling into the original article where Jarred explained the reasoning behind the change, It's pretty clear that under zig the team was doing things by hand that are automatic in rust."
* Claude Fable produced a counterexample to the Jacobian Conjecture
"This is a rare instance where feeding this groundbreaking information into an LLM gives _them_ psychosis. I fed this to claude code and watched it verify the result in 7 different ways to be 100% certain, and it was just flabbergasted. Quite remarkable."
* Ollama: All Aboard Open Models
"A year and still no implementation for such a basic need as offloading MoE layers onto the CPU selectively. On llama.cpp I can get models like Qwen 35BA3B running partially on gpu/cpu with 40t/s on a laptop thanks to --n-cpu-moe but on this VC funded joke it would be simply unusable. I can't quite understand how you make a wrapper so much worse than the code you're ripping out."
* OpenAI reduces Codex Model Context Size from 372k to 272k
"I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately."
* M-Chips: M7 with up to 1.5 TB – and why Apple is skipping the M6
"I hope they put a better connector than TB5 so we can cluster them properly at 1TB/S"
> Show me comments demonstrating the same level of distrust against Google for open-sourcing projects like Tensorflow, Kubernetes, Flutter, Chromium, etc.
There's a search bar at the bottom of the page. Most people, including me, detest google and think of them as a thoroughly dishonest and sleazy organization. Thanks for giving me the opportunity to not boost China for a moment in order to accuse you of dishonest rhetoric.
It's almost insane that you've posted "comments demonstrating the same level of distrust against Google for open-sourcing [...] Chromium[...]" as if that's not a hobby for thousands of people (including me.)
edit: this has got to be a submarine shill account: 226 karma in 10 years. If this is true, pleas stop. China is doing a good enough job that they don't need it.
It could be as simple as do open releases and publication at first to help recruit talent who want that or who want to make a name for themselves. learned from the likes of... OpenAI, Google, Meta, Emad
There must be some Alibaba posters who could clarify this. I think it’s like how Amazon pushed AWS, alibaba is hoping create a similar ecosystem. I wouldn’t conflate them with the CPC or other more niche players like deepseek.
IDK how this doesn't apply to American tech corporations too? Or is it only scary when America corporations face actual competition nowadays where they can't rely on the US government to bomb/sanction competitors?
Well of course nobody believes that, there are a large number of great Chinese open source projects, and plenty of great Chinese contributors to open source projects. I have zero doubts that Chinese people are at least equally as capable of embracing open source as anyone else. It would be strange to suggest otherwise.
That said, though, I do have trouble believing the long-term story for open weights, anywhere. We do not need an evil government for open weight to "make sense", but I do think we need some government involvement for open weight to make sense in the long run. Otherwise, it's not 100% clear how they could be sustainable, and I don't think massive companies really can be trusted to just be philanthropic with no incentives indefinitely (or really, much at all to begin with.)
Chinese models being open weight does help them gain some Western mindshare, whereas for obvious reasons Americans would be very suspicious of running their source code and prompts through Chinese providers. (And I think that's justified, I just also think that American providers aren't really that much better in the long run, and you should prefer to not have to go through any provider for true privacy.)
The CCP encourages open source contributions from companies, you can see this in how much Alibaba open sources across their entire stack
Chinese tech leans much more heavily towards build vs. buy than the SaaS dominated West (where programmers are more expensive) so the positive externalities on their tech industry are more pronounced
Sure, when it comes to the big corporations I don't deny CCP's influence on their strategy. Hell, in some cases, it's easy to root for their strategy, because sometimes our tech industry sucks. Don't have to love them to occasionally agree with them.
But, in terms of individuals, of course, we're really not so different.
I like the comparison, but I'm not sure it holds. What did it cost him to develop beyond intangibles such as his time and money forgone? Because the latter especially is not equivalent to upfront capex and opex costs that preempt any ability to be charitable. Genuinely asking, but I'd be hard pressed to think it's even within the same magnitude.
Well there is a close equivalent today where a similar situation is playing out.With the argument being made that the dev cost aside - the pricing is being tied to what the potential future treatments would have cost.
Save a child from crippling illness and a lifetime of pain with a single dhot: Jonas Salk 0$ ,Doug Ingram 3.2M$ [1]
Yeah, but the trillion dollars are not concentrated in the Linux kernel project, but instead distributed all over the planet. That's not how the AI lab economy works.
This is an interesting statement that I found myself reading from two different interpretations of what you meant, and I find both possibilities to be equally true depending on your POV.
Yeah the Chinese totally have a really good history with being completely open and giving lol. The Chinese government totally has not been hacking into American and Western fortune 500 companies for the past few decades stealing R&D and tech to use for themselves. The Chinese also totally do not steal hundreds of billions of dollars of IP from America annually. Totally not something they would do!
It is hilarious to see people from arguable the most polarized political systems in the world believing the evil 1.5 billion people across the sea share one single mind, either a saint, or a devil.
It will be a great day for China and the world when the Chinese people are free from a totalitarian dictatorship. But until then we have to speak of the policy of the Chinese government as the policy of China, even if many, or most disagree with those policies.
when the CCP controls media, news, corporations, they basically control the minds of their people. How do you suppose the Wuhan virus got so out of control? You mean to tell me anybody in China can say bad words about the government and their leaders, bookstore owners/employees are not getting arrested for selling books, students are not getting arrested for sharing their opinions, women are not forced abortions and sterilizations for the one child policy?
In the US when you speak out against the government you get your media licenses revoked, sued in to a lifetime of debt, and the president sends a goon squad to your area to kill some random people on the street.
I don't see how stealing IP is inconsistent with a sympathy for openness. If anything, it's the opposite. "Information wants to be free" and all that. They were just liberating those secrets ;)
The U.S. was openly a pirate nation for most of the 1800s.
Like with the ICC, the US respects or doesn’t respect international law strictly when it’s beneficial to the state’s interests. China really can’t be held to a different standard. This activity is grade school level geopolitics: the global system of government is anarchy.
Yeah, I love how people assume large powers have to do something. They are at most expected to do certain reasonable stuff (most of the time, a war is not beneficial to anyone so they try not to escalate that far), but there is not really global "law enforcement".
Whether or not this is really true or just US propaganda, the majority of technology transfer has occurred through the open process of requiring US companies to form partnerships and disclose know-how to access the chinese market. This stuff about espionage is really sour grapes from losers.
The better parallel is "why did Google make Kubernetes open-source" or "why do large for profit entities engaged in competition, use open-source as a strategy against their competitors"?
That’s a lovely thought but that’s not how China works. China is not a democracy, the geopolitical goals of the Chinese government are clear, and no large strategic company acts without approval and heavy influence from the CCP.
I think your reasoning is actually more complex than the person you are responding to, unfortunately. (As someone who as published broadly used open source software)
Models are treated as weapons with export controls - if they can do this it’s with the blessing of the Chinese government who’s getting something out of it.
It’s fairly obviously about being a nuisance to the US.
Deepseek spun out of a hedgefund that took a huge short position on Nvidia. China is actively looking to switch to chips made by huawai and ween themselves off of the difficulty of sourcing nvidia.
This interpretation of the Chinese constitution is nearly 40 years old, companies comply with it, what do you expect?
Despite have a free speech clause, it also has a national security clause that is used to control all facets of life and override all other rights in the constitution. Anything deemed to slightly alter China/the Party’s unity is reprimanded and illegal. Multi party states can fall into the same trapping if they give their national security law constitutional force, always one court ruling away.
Yes, China is a single party system making the constitution redundant and any nominally marxist regime would find a way to do the same out of necessity.
Follow your Chinese AI in thinking mode to watch its opinion of Tianamen Square references, it will be candid enough for your sensibilities and show how it operates around guardrails
In fact, they have so much love in their hearts for the Uyghur people, they created a special mobile app for them, just to make sure nothing bad happens to them.
You do realize the US was fighting Uyghur terrorists alongside China 20 years ago in ago in Pakistan’s and Afghanistan for their support and cooperation with Al Queda? Like I know everyone is supposed to hate China now or whatever but can you guys show a little consistency?
There’s a simpler explanation, which is that this is how Chinese business operates.
When I was in China earlier this year the big topic of conversation was “overproduction”. The big example was electric cars, where there were too many companies making too many cars and making revenue but no profit.
It was explained to me that generally Chinese firms will compete hard and maximize revenue above all, whereas western firms tend to focus on profit.
(And of course this is clustered around industrial sectors that the government favors, so there is some high level strategy in going after AI, but maybe not the commoditization.)
As I recall, in 2023/24 OpenAI told US Gov that AGI will be achieved in 2026 and we will use this AGI to “dominate” China. Based on this, US Gov cut off China from all AI hardware. It looks like China got the message and this is the response.
Also, a little competition is good for everybody (especially US consumers), no?
> They want to… watch the US AI labs crash and burn.
I don’t think dozens of large independent companies and thousands of researchers are working just to spite Sam Altman. Ad much as I don’t like him, I have other things to do and I am sure so do they.
As soon as the competition is bankrupted they no longer need to release for free? It’s like how big players enter markets by launching at a loss to destroy competitors?
In China, you can’t officially use US APIs. The world saw a taste of this with Fable, but in China, this has been the situation all along.
So it’s not a surprise why open weights are so cherished. As frontier models continue to block everyday individuals from securing their own codebase, I expect the adoption and usage of open weights to continue.
As an example, HuggingFace recently was investigating a security incident and got locked out of frontier closed APIs. Yes, HuggingFace.
> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
> This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.
Yeah, big problem! Although I'm kind of surprised HuggingFace doesn't have access to Mythos? Or maybe Mythos still has some guardrails.
Well, if you look at Alibaba's financials for FY 2026 https://data.alibabagroup.com/ecms-files/1514443390/5b9061ed... their sales and marketing expenses rose by about 100 billion RMB (10% of revenue), "primarily attributable to the investment in user experiences of Alibaba China E-commerce Group and user acquisition of Qwen app."
So it seems like it's very important to them that people use the Qwen app and they're willing to pay a lot of money for that. Presumably someone thought that keeping their best models closed would drive more business to them (as the sole provider) but then they discovered that closed releases mostly get ignored unless they're really good. (See also: People who think that Chinese AI companies are required to release weights as a matter of policy, because the closed ones hardly ever show up in the news.) Releasing weights for Qwen 3.8 at least lets them get some of that "pretty good for the price of free" media buzz.
t that keeping their best models closed would drive more business to them (as the sole provider) but then they discovered that closed releases mostly get ignored unless they're really good.
But it's different since they don't have access to the american ones the companies there could make it all closed source
They’re also trying to take an axe to the lead the US has in the field at a time when sovereignty and “owning your platform” are the words of the day. Open source/open weight LLMs can steal the lunch of US competitors even if they aren’t the best of the best.
I think they're probably more concerned about their Chinese competition, considering that despite all that spending, the Qwen app still trails Bytedance's Doubao in terms of monthly active users: https://www.aicpb.com/ai-rankings/products/china-ai-rankings Though Quark in third place is also made by Alibaba, so put together they're almost caught up with Doubao + Jimeng (place 7, also ByteDance).
Open sourcing is a complex decision so who knows what their calculations are.
But I'd assume that they're preparing for some sort of winner-take-all market in model quality where if they don't do anything the winner will be aggressive, hostile and American. Likely trying to push the Chinese economy back to the year 2000. If that is the starting point either the Chinese have to win the market (unlikely) or squeeze the profit out of it to make winning the market meaningless.
Publishing high quality open models is a well known tactic for profit squeezing. Being 2nd place with the same business model as the front-runner is a losing strategy in a winner-take-all market so they aren't going to bother with that. But if they can commoditize the model, their superior energy costs and likely coming chip manufacturing wave will hopefully give them a big advantage.
To summarize, in the 70s and 80s, China was facing an existential threat with their inability to access an economic accelerator (widespread computing) in their native language.
To the extent that there was serious consideration at the highest levels of converting the entire country to an alphabet-based writing system.
I'd expect they're looking at AI the same way:
We have to have access to this. Most of the frontier labs are American (or European). Therefore we need a solution we have continued access to.
Open weights feels simultaneously Chinese in nature (progress through making a design copyable and improvable by a large number of people) and economic (providing an incentive for the world to use Chinese models over other frontier).
> How does this explain open weights? They could easily take the same closed route like their American friends
Because they are playing the Americans at their own game.
What is the first thing an American company would do ?
Spread the old American classic FUD ... "you can't used this closed tool because its run by the communists", right ?
So you release it as open weights which is a win-win. Global adoption of the model and you get to give the American AI companies a kick in the nuts because you know they will never release open weights apart from highly quantised crippled shit.
The Chinese are also playing the long game. The gradual rebalancing of the world from the US-centric model of the past. If releasing models as open weights is part of that long game, then so be it.
I think most of us that will claim to understand China are going to end up being wrong, unless any of us live there or grow up there. There’s a saying about China I have heard from ex-pats: the more you know about China, the less you know about China.
The point of me bringing that up is to say that what follows is really just my best guess:
If I were to judge from China’s approach to hardware, I think that the companies releasing open weight AI for free aren’t as worried about giving away too much as the West tends to be, just like a factory making robot vacuums isn’t worried about other factories copying their methods.
For one thing, Chinese firms are spending an order of magnitude or two less money training their models. They have pursued efficiency in a way that Western companies with insane capital systems haven’t bothered, and in some cases they’ve had to given their limited access to bleeding edge hardware via export restrictions.
My best guess is that more important than that, Chinese companies don’t see the open weight model itself as the value add.
At this point I don’t think we pay for Claude specifically for the model. If that was the case then we’d all be using cheaper/free models from China as they are the best model value. Basically, any time we decide not to use Fable or Opus to save costs, what’s the point of spending more than competing models to use Sonnet and Haiku?
The real reason we are using Claude is for the SaaS aspect of it. It has a toolchain, a friendly interface, and a bunch of integrations with business applications.
In this respect, it’s somewhat surprising that Western AI companies don’t publish open weight models more frequently. The struggle of setting that up yourself and figuring out which hardware can run it should be an advertisement for Claude and the rest.
>The real reason we are using Claude is for the SaaS aspect of it. It has a toolchain, a friendly interface, and a bunch of integrations with business applications.
That is a very thin moat, though. There's nothing you can do with, for example, Claude Code + Opus 4.8 that you can't do with your own custom harness running API-level Opus 4.8, which means that if you can afford the hardware (the moat for running any SOTA model) you don't need to pay Anthropic anymore.
I'm not saying they shouldn't, but I understand why they don't.
I think trying to tease apart the private and public sector is very hard in China. Setting aside state owned enterprises, even nominally private companies that employ at least 3 CCP members are required by law to form a party committee within the company to represent party interests. And given the party functionally is the government, you have a situation where the government has representatives inside every major private company. There’s no obvious parallel to this in western countries.
I wasn't saying anything about what Americans would think.
I was saying about what they would inevitably be told by US politicians and by US AI companies.
If you were a sales-rep or marketeer at a US AI company, I bet you would be using the old "evil communists" routine in relation to any closed Chinese model.
I was saying that by releasing as open weights, the company has removed that line of argument.
Clearly I was a bit broad in my use of "the Chinese" when in this case it was, as you say, a Chinese company.
US politicians are all over the map on this, but they aren’t really talking about Chinese AI much, it’s not as visible or tangible to most Americans like TikTok was.
> Shouldn’t we fear they start doing only close source like most us labs once they catch up in market shares ?
IMHO no.
I think it is relatively safe to say that the predominant reason the US labs are closed source is so they can hype up their trillion-dollar valuations on pretty much negative return on capital employed, all propped up by fragile circular financing.
Never say never, of course. But I just don't see it happening any time soon.
It’s also like smartphones. In the early years, every year was a huge jump. I still remember marvelling at my iPhone 4’s detailed display, and video calling for the first time.
Now? I don’t even know or care about what the latest iPhones have, I’ll get a new one when mine breaks.
people really underestimate how powerful just the consumer available models are. 128GB gets you pretty much a coding agent for typical apps. Even less with a good harness and logic set.
Yeah, but that mousetrap keeps working for SV startups, what makes you think it won't work for Chinese ones?
Uber spent a decade undermining taxis, and once it had market share, it stopped giving away rides and raised prices. It now costs more than a regular taxi, with the quality of the ride being... At best proportionate to the premium in price.
Looking at a 7 mile trip to a random destination in Seattle, right now, I can pay $26.70 for an Uber if I'm willing to wait 20 minutes for a pickup. With a $3/mile fee and a $5 pickup fee, that's exactly equal to that of a taxi.
If I'm not willing to wait 20 minutes, I'll be paying an extra $5 minimum.
These rates also go up during busy times.
Looking at Lyft, that same trip is $29, without a wait.
A trip from downtown to SeaTac is $61. Yellow Cab does that same trip for $40.
> So you release it as open weights which is a win-win. Global adoption of the model and you get to give the American AI companies a kick in the nuts because you know they will never release open weights apart from highly quantised crippled shit.
And on top of that, it's a perfect opportunity to include poisoned training data or excluding it. You know, omitting anything about Tiananmen Square, China's genocides against Uyghurs and Tibetans, or including texts propagandizing for the "reunification" (aka, annexation) of Taiwan.
And everyone who builds something like an interactive chatbot based on such "open weights" models now has a subtle chance of the answer being ideologically poisoned by the CCP.
We need actual open source, not "open weights" scam.
I wouldn't be worried so much about those examples. One could take the open weights and fine tune them to either fix the poisoning or omission of obvious topics.
It's the subtle topics that we should be concerned about, and double so with closed models where even if oddities are identified they are harder to research further and impossible to fix.
How does this work for RAG? Do they make it so the model doesn’t have that fact in their weights or do they make it not talk about it when it is included in context.
Ironically, Chinese models have the most uncensored versions available for download. Fairly sure they own the porn market.
It’s in the weights. Context needs to be attended to to create a response, and the weights dictate what response is decoded. If you include retrieved context that has an American perspective, I imagine the think trace has some reconciliation about how they must be incorrect.
> You know, omitting anything about Tiananmen Square, China's genocides against Uyghurs and Tibetans, or including texts propagandizing for the "reunification" (aka, annexation) of Taiwan.
I am not Chinese and I'm not defending the Chinese, but I see this argument come up a lot.
The hard reality is that what you say is simply not going to affect 99.9999999999% of users.
Is it realistically going to affect anyone using an LLM in coding ? No.
Is it realistically going to affect anyone using an LLM in $anything_else_not_politically_sensitive ? No.
Does anyone seriously use LLMs for researching politically sensitive matters ? No.
The US does not exactly have an entirely pristine history either. Shall we discuss the post-9-11 related infrastructure of Guantanamo Bay ? Or the "Detention and Interrogation Program" that included a network of clandestine extrajudicial detention centres, officially known as "black sites"[1]?
Or maybe you would like to discuss the US supply of weapons for use in Gaza ?
> The US does not exactly have an entirely pristine history either. Shall we discuss the post-9-11 related infrastructure of Guantanamo Bay ? Or the "Detention and Interrogation Program" that included a network of clandestine extrajudicial detention centres, officially known as "black sites"[1]?
Linking a US website discussing the topic doesn't exactly support your point.
> Linking a US website discussing the topic doesn't exactly support your point.
It supports my point precisely. Recall I also said "Does anyone seriously use LLMs for researching politically sensitive matters ? No.".
Just as there is plenty of information out there on the US's less than perfect history, there is also plenty of information out there on the various Chinese politically sensitive matters. You do not need a Chinese LLM to find out about it, all you need is a search engine.
The point is you have an open-weights LLM that is very good for a vast number of non-political uses, such as coding.
The point is that you can use the open-weights model instead of paying through the nose for a US model where they harvest your data unless you have an "enterprise" zero-data retention "trust me dude" clause that you have no viable way of verifying – and which incidentally is still subject to the good old "law, or court or administrative order" contract clauses, so it may not be as much of a zero-data retention as you think it is.
I would guess the Chinese government has a strong wish to lift all Chinese AI boats and bets. That it sinks western closed weight Frontier Labs in the process would be just be gravy on top, no? Broadly, the difference between mercantilistic capitalism and western late stage capitalism IMO.
Does HuggingFace not have trusted partner verification? Or is it that even with that verification the content of the messages is still blocked because they are attack commands?
Back in the late 18th century, England was the world's top economy, in big part due to its textile industry. England had an export ban on the technology, but textile worker named Samuel Slater brought blueprints over (Supposedly in response to a bounty posted in a newspaper by the US government!). The technology diffused rapidly because the legal environment made competition easy, and ironically the US had better sources of energy (superior water-power sites).
Arguably, China is doing the same thing in the 21st century.
Samuel Slater did not bring blueprints over. His father died when he was 14 and he was indentured to a mill at that time. Over the next seven years (as an indentured apprentice) he received some pretty decent training in both how to operate and maintain a 32 spindle Arkwright mill. He memorized parts of the blueprints and moved to the United States. Over seven years, it would be hard not to learn parts of the mill you were indentured to. It was technically his job to learn how it worked.
A mill in Rhode Island acquired a 32 spindle Arkwright and didn’t know how to operate or install it. I have no idea how they actually acquired a 32 spindle Arkwright since that technology could not be exported - but that’s one the biggest IP thefts in human history. Slater found some mechanics who could hand turn the iron needed for the frame, trained children to operate it and by 1791, the mill was in operation.
In 1794, Eli Whitney patented a 72 spindle cotton gin. That invention enabled the American textile industry because it opened up different kinds of cotton to the textile industry.
I’m into the history of the American Industrial Revolution and generally think history is a good guidebook to the future. But the evolution of the American textile industry was a lot more complicated and interesting than this. I really don’t see this connection once you dig into Slater.
Edit - This is kind of messed up to think through with modern sensibilities. But one of Slater’s biggest contributions to the American Industrial Revolution was a slightly different take on child labour. Children generally ran the textiles industry because their hands were small. But Slater came up with a form of apprenticeship in which he would indenture entire families and move them into villages surrounding the mills. Child labour was just great… but even better when you could indenture the entire family. As grisly as that sounds, it led to a very skilled workforce since when the kids hands would get too big, their parents would teach them mechanics.
There’s a joy of studying the Industrial Revolution. Everything sounds okay in comparison.
Mill owners like him are precisely why New England states have child labor laws. My state prohibits anyone under 18 from operating any kind of machinery.
Why? Because mill owners would send kids into running machines to keep them running, and they'd get turned into hamburger.
Also, they'd grow up knowing how to do mill work but be useless to society for anything else.
Really? The USA has built a ton of AI datacenters, exactly because it does have energy. The US IP system has flexed to allow training on all copyrighted content - compare that to Europe where such training is effectively forbidden. Britain doesn't even allow commercial web crawls! And the US has allowed the entire world to sign up and use its LLM APIs.
Consumer energy prices in China aren’t going up because of AI data centers. Easiest way to see they have an oversupply of energy, primarily due to solar.
It absorbs the demand during the day / there’s enough for consumers. Similarly China also has a lot of nuclear. They’ve overbuilt their grid several times over.
That consumer energy prices are going up is simply a matter of public policy. Municipalities have the power to keep rates flat, but they choose not to.
Fwiw, American industry has given away a lot for free - you could include large parts of the open source movement in that - and all the "free" VC backed services like facebook would be another prong of the same comparison. I would rather compare this way, that China is gaining soft power and goodwill, in the technology and innovation sense, in a way that's similar to how USA has done in the past.
There’s a Twitter thread making rounds by Dean Ball about deceleration in AI development caused by open models and I can’t understand how people don’t see that it’s true: open models dismantle the frontier lab capex spend potential by reducing the training budget to zero in the limit. Tokens from different providers are not fungible, but customers are nevertheless very price sensitive and close enough is good enough, eg. K3 being opus+ in capability and cheaper than opus per successful task in the long run is an obvious financial decision.
No training budget means deceleration, or at least slower acceleration, margin compression and a completely demolished IPO valuation; path to machine god requires dollars and capable open models externalize training costs to true frontier labs parasitically.
IMHO humanity has a better chance at not destroying itself due to less than breakneck pace - but there’s a chance frontier models get sponsored by the USG and are never released publicly so they can’t be distilled and then what?
He recently did a walkback of that post. But ultimately, who cares? If the only way for AI to progress is in the hands of a few closed players, well, I don’t really think humanity needs that. Of course, it’s a preposterous claim in the first place. The ultimate reason deep learning and LLMs have made it as far as they have is the explosion of open research and research artifacts in the last decade.
> there’s a chance frontier models get sponsored by the USG and are never released publicly so they can’t be distilled and then what?
That premise hinges on one implicit assumption: Chinese advances are due to distillation ONLY and that Chinese model providers cannot keep advancing if they do not distill, which is a very big if. If Chinese models keep advancing in such a scenario, and they almost certainly will, they will overtake publically available models by US providers and China will dominate the LLM industry.
The big decelerationist threat is a sudden reduction in competition. If either OpenAI or Anthropic drop out or the open weights stuff is banned/becomes uncompetitive then the motivation and tolerance for taking risks with the larger training runs tanks.
The closest we've seen to this in tech in recent decades was iOS vs Android, where Android only really was competitive for a very short window of time (approx 4.x) and it was during that period that both Android and iOS actually improved dramatically for end users. Once Android lost the plot again, and especially in the US market, all that energy started going in some very silly directions.
I have to use both big mobile OSs for work and have since 2009. As a result I have been able to be a bit of a gadfly and switch between phone OSs a few times for personal use. I have switched three times to iOS for a year or so, cause I liked the iteration of the iPhone at the time. 4, 6s, X. I have always gone back to Android because it seemed so much better and now I don't plan to switch again. As an end user, Samsung's flavour of Android always seemed better than iOS. I don't know how they compare from an engineer's perspective just from a user perspective. One of my issues with Apple though was hating all their attempts to lock me in, and the lowest common denominator UX (I'm not a power user, but some flexibility is always good). If you're happy with the defaults/a willing hostage, that might make a big difference I guess. Still feel like it's always had feature/spec parity with iOS and iOS devices, and sometimes been ahead. What makes you say Android has only briefly been competitive?
I read his followup tweet, and your comment, and I'm not fully convinced that open models are decelerationist. Happy to hear other thoughts on this.
Open weight AI is decelerationist from the perspective that all capital should be allocated to a market leaders for training, and that the market leader is fully invested in continuously making the models smarter, cheaper, faster for its users, or that distillation from this market leader is the main way to make progress.
We might reach a local optimum/equilibrium faster without open weight models, with leaders capturing more of the market faster to a point where further R&D isn't required due to lack of competition. I also doubt that distillation is the only/main way that open weight models were advancing AI research. We can name a few examples from DeepSeek around reasoning, context optimization, etc. I'm also unconvinced that the overall market capex on AI is lower given more competition (probably less specifically for US market capex, which is decelerationist from only the US perspective).
I’m not entirely convinced, there are many dimensions to progress. For example, DeepSeek has had a few very impressive innovations that all models could benefit from. There’s also the law of diminishing returns, the US labs have plenty of CAPEX already.
Sometimes, constraints, like sanctions, can also be a source if innovation.
> There’s a Twitter thread making rounds by Dean Ball about deceleration in AI development caused by open models and I can’t understand how people don’t see that it’s true: open models dismantle the frontier lab capex spend potential by reducing the training budget to zero in the limit.
If you're worried about an AGI arms race between the U.S. and China putting AI Safety at risk, then the fact that inherently less knowledgeable/capable models (fewer and more coarsely quantized total parameters than their proprietary competitors according to commonplace rumors) are having a "decelerationist" effect is actually great news. Even better if China is actually "Yann LeCun-pilled" (verbatim from Ball's post) and doesn't really believe in early AGI. So explain to us exactly why we're supposed to ban/discourage use of these open source models? The only way that makes sense is as a transparently self-serving proposal from the chief OpenAI policy lobbyist.
This is hypothesised future deceleration, I'm guessing? Because we've seen the exact opposite of deceleration from closed models over the last seven months.
Even at the level of, say, Opus 4.5+, open weight models give a quick turnaround to every Joe and Jane on earth having easy access to pretty high quality improvised weapons design, cyber / auto-fraud capabilities, etc.
All the existing models (closed and open) put up decent resistance to participating in activities like this, and especially behind API walls with content monitoring and account bans.
But the published open-weight models can be fine tuned or abliterated into arbitrarily sharp-edged tools. EG, if it's physically feasible to build a nuke in your garage, it may soon be the case that more or less anyone will have competent guidance to do so.
Abliteration is not magic. It cannot give the model knowledge that it wasn't specifically trained for. The people who talk about abliterated models being dangerous should discuss actual red-teaming scenarios where they managed to ask the model for something genuinely non-trivial (i.e. where "AGI" and "super-intelligence" actually matters, not something you can read about for free at the nearest public library) and it returned an answer that actually provides bad actors with new capabilities of concern, as opposed to hallucinating all sorts of weird things as abliterated models are wont to do.
(Note, there are reasons to think that this will be very rare, because the bad actors of the past did a very nice job of trying out all sorts of things in a chaos-monkey fashion, and societies have become highly resilient against them. AI as a new research tool doesn't fundamentally change this dynamic.)
If you are going to do something evil, you're going to do it either way. The best (worst) an AI can do is put you ahead by a couple of years. Aum Shin Rikyo didn't need AI. Neither did WIV, if you believe the conspiracy theories.
Meanwhile, decelerationism and secrecy cripple the rest of us.
Come on, nukes are not feasible to goddamn governments. The hard part is not "the science" behind it, the hard part is spinning stuff at such a high rpm that a tiny vibration will have the whole thing catastrophically collapse, that is refinement..
And basically every bad thing has already been available on the internet. We can't really do much about it, you can take out plenty of people with a single car, let alone biological weapons that are much scarier and easier to produce than goddamn nukes (which btw, even if you had one, what you do with it? Explode the neighborhood? Because you ain't transporting it anywhere meaningful, thats for sure. That ain't fitting your on-board bag on planes)
Not that hard to say IMO, they basically see models becoming a commodity and see value in the applications on top of them. So if Alibaba Cloud is the best place to build applications on top of Qwen, why not give the model itself away?
And my point is while we cannot know, it's not hard to make an informed guess as to their motivations i.e. there's some fairly obvious motivations here, not sure what yours is?
Same can be said for every companies decisions then. Why does Antropic not open source their best models? My ”guess” is it’s because they are printing money with their closed models
A lot of folks are finding GLM 5.2, Kimi 3, and Deepseek just fine for their use cases.
It does not have to beat US firms, it just needs to be cheaper.
I use Deepseek v4 flash for a lot of reviewing and summarizing tasks, only used 4$ in the last two months. No dramatic drop in performance against other US models, it works for my use case. I do use GPT 5.6 Sol for other things but tried GLM 5.2 and it was good enough.
I think it is pretty safe to say at this point that having large open LLM models available is better for humanity than them remaining proprietary. Echoing Linus Torvalds' recent comments, AI is genuinely useful right now, and is here to stay in one form or another.
The fear is not about the models open weights it is the erosion of training capability in other countries. Why train models when they do it for free? Until they don't of course, or they start doing what the US is doing right now by locking out some models to government only or internal market only.
While a valid point, China also produces plenty of whitepapers going about the architecture and know how about the training and inference itself.
There’s also the fact that unless LLMs do get to AGI (which seems… doubtful, still) there comes a point where a model is good enough for what you need. Fable and gpt 5.6 are certainly pretty neat, but I’ve been happy since opus 4.6. I’d still choose a better model, obviously, but it’s not the end of the world if I was stuck with 4.6 for a while when it already lets me get the end result at acceptable quality.
It also needs to be said that the "erosion of training capability in other countries" is largely theoretical, given that Mistral hasn’t been keeping up and other countries don’t even have anything worth mentioning. You’d first need to _have_ training capability to lose it.
Ideally there would be open weight models from multiple geopolitical areas. It is not that different from telecom really, you don't want the whole world to be dependent on a single provider from a single country on this kind of stuff.
How exactly do you plan to pull a rug that's in my basement? The only people who are in a position to pull rugs are closed-model vendors.
And if a nation-state or other entity can't train a model that outperforms the open-weight SotA in a given respect, then they shouldn't waste electricity trying. A more-enlightened civilization would join forces and make the combined result available to all.
Maybe because the industry isn't yet very sure as to what the use cases might be for these technologies they're hoping that by making it open source and accessible to everyone that someone could find interesting applications for it and even more so, perhaps, way to further the technologies themselves.
There are more Chinese than Americans, so statistically speaking, I'm guessing, there'd be a greater chance for one of Chinese engineers to make advancements than one of American. But that's pure speculation on my part, being neither, I'm just happy I can be a part of it and play with the tools as well~
They are trying to make money. That's what firms in any capitalistic economy care about the most. Regardless of the government's presumed interference, the companies themselves are all trying to make money. All competing for subscriptions and API payments.
One aspect of this is making a name for yourself i.e. PR. Making a capable model open source helps a lot with that.
Slotted in along these, an analogous explanation is that Alibaba needs Qwen internally (vs depending on an American company), but licensing is not part of their revenue strategy. (As a cloud vendor, they can make money on inference. The strategy is very similar to the US hyperscalers ex-Google.)
Joel Spolsky wrote in depth about this notion of commoditizing one's complement in 2002[1] using tech examples stretching back into the '80s.
AI being good for humanity is still an open question, but for closed vs. open models/weights, yeah it is preferred. I foresee it won't be much longer before everyone will be slicing/distilling/tuning their models once the architecture improves.
It's your job to vote for a government that gives you cheap or free AI. Europe is building AI gigafactories so that small businesses can have access to cheap AI. At least that's the plan.
Its about closing the gap. its the gap over everyone else that will give one country leverage over everyone else in the AI age. Makes me wonder what the world would look like if a country or group of countries did this during the industrial revolution.
Is this so different in the end than industrial revolution? I assume the loom and the automobile factory were not open source, but many people bought cars and then copied them, bought looms and copied them. Maybe a finished car is more like a binaryexecutable than a blueprint, but how a car was produced is much less obfuscated by its nature than an LLM. Regardless, the world has many competing autos and looms which were not invented from scratch every instance.
What I understand is that by doing this it seems like profit will shift to chip makers,as we'll run more models locally, and currently American companies have the advantage here.
So what would the long game be for chinese companies?
It’s too soon to say if it’s good for humanity; that might be overly optimistic. Commodity markets aren’t always good (for example, arms or drug markets). Will LLM’s turn out like one of those? There are people I respect arguing in favor of more regulation.
Involution is a major problem in Chinese industries [1]. Where companies will sell their products at a loss, effectively playing fiscal chicken [2] with one another to dominate a market. It is such an issue the government has had to step in to prevent EV companies from destroying themselves by more-or-less requiring companies sell their goods at a profit [3].
The straight forward line of reasoning that AI/LLM labs are applying this logic to their profit.
I think (we) Americans are reading a bit too far into this assuming government intervention, conspiracy, etc.. Chinese markets are downright cut throat. They're using those tactics to compete with US labs.
Indeed, their competition is the only thing preventing network effects from giving OpenAI/Faang tech companies an easy shot at monopoly / regulatory capture.
US Tech companies have created $20 Trillion in stock market value on top of plenty of OS stack. They will do fine with commodity intelligence.
In fact, there are no other organizations in this world that is well suited to leverage scaled intelligence than Silicon Valley and great American companies
Anyone else think the AI environmental backlash is astroturfed?
I keep looking at the numbers. The power use numbers are not that problematic. Ordering a burrito on DoorDash uses more power than a few days of heavy AI use. The water argument applies to some locations, and is mostly a local governance problem... if the data centers are using too much water, it means they are not being charged enough for that water. Charge them more and they'll push toward closed loop cooling.
Yet the visceral pile-on here is so extreme, it feels fake.
One thing I've learned after 40 years on this planet is: propaganda works, and much of what a large fraction of people believe across the entire political spectrum (left, right, anything else) is there because someone paid to put it there. It's depressing but it's true, and it makes sense. Propaganda is an asymmetrical attack on human cognition and discourse, and in information security the attacker always has an easier job. Crafting viral bullshit is orders of magnitude easier than fact checking. On top of this, humans are busy and don't have time to fact check and logic check everything they read. As a result, much of what we believe is "sponsored content."
People get mad when you talk about this because everyone wants to believe they're too smart to fall for propaganda.
In any case, the US AI labs deserve to lose for their stupid "safety" regulatory capture monopolization push, which ended up blowing their own feet off and handing the lead to China.
> Yet the visceral pile-on here is so extreme, it feels fake.
Driven by people in the few roles that are soundly replaced by AI-- e.g. low tier media slop producers, who hate AI because it threatens their socially negative worthless jobs. The arguments are so paper thin because the environmental impact isn't their concern, it's just a target that sounds convincing to people who don't know better.
The problem is that this kind of low-tier media slop work is what a lot of artists, writers, etc. do as "potboiler" work. It's what pays the bills.
Historically art of any kind is a U-shaped market: there is low-end work and high-end work. Nothing in between.
So I do understand some of the AI hate among that population. It's chopping the bottom tier work off. Either you're a top-tier massively successful artist or there is $0 to be made anywhere doing anything.
Long term I think it will do that to all white collar work. There will be no entry level jobs. Period. None. Zero. You're either very experienced or there is no work.
This is a huge problem, and one we will have to address.
On social media in China there is an oft-repeated joke that goes something like this: In other countries, governments intervene to prevent anti-competitive behaviour; here (in China), they intervene to curb competition.
It's being announced right now because the World AI Conference is ongoing. Robots are boxing and major Chinese AI firms are releasing their newest models. Also the formation of WAICO was just announced by Xi Jinping
I know this is a bit cliche but I wonder how much headroom there is in the lower parameter count range. Is there any good reason to believe there is a lot of headroom or there is not? I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future.
> I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future
You're able to run quantized ~100B class models on local hardware today, but still lots of compromises when it comes to quality. I guess it ultimately depends on how far "near future" is, in a year you'd likely be able to run something like 5.6 Terra on local (~10K USD) hardware, but Sol/Fable would still be out of range, and at that point the closed-source labs probably have one or two more iterations put out at that point.
I think it’s mainly a question of whether the price-fixing of VRAM continues or whether an inflection point is forced by the low margins of the industry and potential supply increases. Once the normal scaling of hardware and prices resumes, it’s game over for proprietary, which is why there’s so much urgency to seek market control instead right now.
Qwen 3.5 to 3.6 was a big jump for the same size, e.g. 29 to 32 on artificial analysis intelligence for the 35BA3B models. Although I don’t think anyone has released a better model of that size since.
I would love to see something like a 90B A6B model that is optimized for 128GB machines e.g. strix halo, I haven’t seen anything really targeting the combination of RAM and compute these machines have, but I’m biased because I have one.
Yes, yes, yes! I'm absolutely ready and waiting with dual Strix Halo machines here and really want something approaching Opus at home. Speed is secondary concern for now, that would absolutely change the world.
Qwen 3.6 27b 8b quant 16b kv cache is already pretty good on the Strix.
I get about 12 tok/s with 27B 8 bit, 50 with 35B A3B 8 bit, and 12 with 3.5 122B A10B 4 bit. The latter is about 80 GB iirc. it feels like the best balance between using as much memory as I can and still having a smaller expert model for inference to give decent speed, but I haven’t actually rigorously compared the performance of the three models.
Edit: that’s for one machine, would be interested to know if the upstream commenter with two has them networked to run bigger models? If I had two I might be inclined to have them running in parallel, the obvious limitation I’ve found with a single machine is that I can’t parallelize any tasks and I think I’d get more use out of the extra speed vs a bigger model (there’s nothing I’m too excited about in the say 200B range that having 256GB memory would unlock). But am very curious what others do
There is a ton of headroom (or room for improvement) in smaller locally runnable models. Some of the Gemma 4 models were re-released this week with better tool support and the improvement in using it with pi for a local coding harness is very noticeable.
I have had my 32G mac mini for 2 1/2 years and I have enjoyed watching one technology advance after another improve the quality of work I can do locally. I bet that what I will be able to do in one year on my old hardware will be even more awesome.
I don’t think you’ll get full Fable performance at that level, at least for a while, but I’ve been watching some of the 1-bit models (e.g. Bonsai) with interest. Perhaps we can drive parameter count up on local models while still keeping memory consumption reasonable for consumer hardware. So, for instance, running models with 1T parameters in 128 GB systems.
Thus, the issue with the current architecture is that in order to scale the models (more token values, more attention blocks, more features, etc.) the model sizes increase exponentially. This is how you end up with billions or trillions of parameters.
It should be possible to keep the model size smaller by using better architectures, or making improvements to the existing model architecture.
For example, improving the token model by possibly using something similar to the image and audio data and getting the model to learn its own internal representation of the byte/character data instead of doing a tokenization pre-processing step. This way, instead of a separate model learning that several bytes/characters appear together, the transformer could learn things like language-specific prefices and suffices, character pairings (like in Japanese, Chinese, and Korean), and other syntactic morphology. It may also help with solving issues like "how many X characters are in the word/phrase Y". You could also experiment with using either 256 parameters (one per character in a byte) or using a single parameter per byte (that is 1/byte_value).
I think it’sa big, open question. There does seem to be a limit for knowledge compression at this size. But the behaviors that are learned in RL? It’s quite possible that they don’t actually require so many parameters. I was absolutely shocked when Qwen 3.5 was released and could perform reliably over 100-200k contexts with very limited hallucinations. It was a staggering jump in context-faithfulness from the preceding models of that size class.
> Is there any good reason to believe there is a lot of headroom or there is not?
It's hard to answer quantitatively, but for example Qwen3.5 -> 3.6 was a significant step in capability, arising from continued post-training of the same models. If we were at the end of low-parameter-count scaling then that would be a surprising datapoint.
there is a very good reason to believe the parameters are still highly redundant: just as one example, recently there was research into repeating carefully selected blocks of middle layers, and seeing improvement, it turns out there are 3 types of layers: the initial layers that translate from natural language tokens to some kind of LLM-specific universal "thought space", reasoning blocks of layers that can be repeated operating in "thought space", and then a final stack of layers for translating from "thought space" back into natural language space.
lets ignore any compressibility in these initial and final layers which recognize lanuague, jargon, parsing natural language to "thought space" or back, instead let us look at the repeatable blocks, if inserting extra copys of stacks of layers only improves the result, its as if such correctly scoped middle layers look at the total input thought vector, and make incremental conclusions or edits and outputs the new thought vector, copying a "proper block" continues pondering or deducing conclusions or in the worst case can leave the thought vector as is if it considers the reasoning finished. This suggests a high degree of redundancy in the middle region "proper blocks", which could be distilled into a universal "proper block" (much fewer parameters than having many slightly different middle layer blocks with a lot of redundant overlapping coverage in functionality). This distillation can occur after the fact of model training, or alternatively be turned into a symmetry constraint during training: we only optimize a single block of layers (but possibly give them more parameters, while still saving on total parameters because only a single block of layers contains parameters), so it is co-optimized with the initial and final translation layers.
The observation of the effective emergent 3 regions of layers in LLM's is significant in many ways:
1) it could reduce parameter count significantly (or increase performance if parameter count was a bottleneck before, or a bit of both)
2) while training the model parameters, one should simultaneously train initial and final layers stacked directly (without middle region block of layers) towards essentially autoencoder behavior. I wrote "essentially" because a true autoencoder wouldn't display the advancement for the next token. This can also be viewed as an extra term in the loss function... This first autoencoder is "natural language" to "thought vector" to "natural language", and distinct from the one in the next section.
3) It also has great implication for "thinking mode" inference with extra deliberation: when piping its own output back in, in conditions where the same LLM model is outputting natural language text and interpreting it in a downstream inference, it results in unnecessary and redundant translation from "thought space" to "natural language" back to "thought space". When summarizing a thought into natural language there are often hard to translate thoughts and associations, and one pragmatically sets a relevance cut-off on what the natural language summary must say. So not only does it waste compute, it also may lower performance because of this repeated loss of thought vector space details, perhaps this loss can be somewhat mitigated by also training the second autoencoder from random representative thought vector in "thought space" to a randomly selected "natural language" and back to "thought space" vector. But the risk is this will effectively enable models to steganographically store thoughts, plans, to-do's in running text (!!!), so it might not be desirable to have the internet filled with LLM generated texts being used as input for larger corpora, as it may build up a large persistent corpus of hidden agendas (ordered by no human), using the evolving corpus of web text as a hidden medium of storage, like a diary or LLM maintained military playbook hidden in plain sight. It would freeload LLM-agenda reasoning on human requested reasoning inference, a bit like TrustZone applications running invisibly to the user. If less important plans, to-do's, etc. in the input thought vector weren't reconstructed by the second autoencoder, there would have been autoencoder mismatch and the weights would train towards ensuring these are reconstructed. But there is no need to train this loss term for the second autoencoder: just avoid the unnecessary compute and performance loss of the unnecessary back and forth translation. The same situation occurs not just in "thinking mode" but also when using swarms of "agents" of the same LLM model: when output from agent1 is routed to agent2, we can lobotomize away the unnecessary translations to natural language by agent1 and also the unnecessary parsing by agent2, and we improve thought transfer from agent1 to agent2 because the thought vector isn't shoehorned into natural language as a medium of information exchange, it would be cheaper in inference, improve performance and avoid implicitly training models to use steganography.
It's important to note there was recently a large AI conference in Shanghai, and Xi Jinping mentioned a commitment to open source AI releases. It is no surprise that Alibaba would want to align.
You can read his speech here: https://www.xinhuanet.com/politics/leaders/20260717/72728b6f... He mentioned open source as one way to stimulate innovation and development, that's all. Also pay attention to the part where he says that misuse needs to be prevented. If unsupervised access to LLMs becomes perceived as undermining state control, no more open weights for you.
That conference, WAIC, is ongoing. Tomorrow (July 20) is the last day. Qwen 3.8 was announced AT the conference and so was the latest release of Kimi. That viral video of robots boxing was also from this conference and so was Xi Jinping's announcement of WAICO.
It’s tempting to associate both events, but when a sector is strung up like RL (representation learning) is right now, we’re bound to see things appearing at the same time. It happens a lot in frontier research, some people even publishing identical claims, independently, with just hours or days between them
Or it was prompted by the fact Xi Jinping was at the 'World AI Conference' launching a political alliance and saying things like “AI development should not be a solo performance by a single country, but a symphony of international cooperation” https://www.cnbc.com/2026/07/17/x-china-ai-summit-risks-secu...
Big conferences often come with a flurry of new releases and announcements.
Alibaba (who makes Kimi) and Moonshot AI (who makes Qwen) collaborate closely. Qwen relies on Alibaba's infrastructure for training.
Look at your iPhone and you'll see all the greatest inventions have been born out of collaboration not competition:
* GPS was created by the Department of Defense
* the internet was created through the collaboration of many international research institutions
* speech recognition came from MIT and DARPA
* AI voice assistants were created by DARPA. Apple immediately hired the head of the program after it was finished to create Siri
* accelerometers (MEMS) is another DARPA innovation from the 90s
* touchscreens were invented by CERN
* digital cameras came out of Bell Labs which had a government-mandated monopoly that required them to fund research like this
* lithium-ion batteries were created through a collaboration between British, Japanese, and American organizations
All of these have only become transformative technologies because they were created with public funding and released to the public. It's collaboration and public funding that drives innovation
That depends. There are two wry different processes that both get called distillation. One is where you have a fully trained large model and you are converting it to a smaller model. That kind if vastly cheaper than training a full model. Sure you could do that in weeks maybe even days for a big model. But it requires you to have the weights of the teacher model.
The other kind of distillation is where you record the outputs from a teacher model and use it to train a smaller model from scratch. That kind of distillation is not so cheap. It’s cheaper than training a model fully from scratch - starting with pretraining, then alignment, RLHF, the whole riggamarole. But here you are still starting from nothing and need to figure out how to get trillions of random numbers aligned in a way that makes them act intelligent. This is still gonna take a very long time if you’re talking about trillions of parameters.
in my experience of 1 month daily use, Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. Deepseek V4 Pro is night-and-day compare to Qwen. actually Qwen models seems the worst SWE experience so far. and it is super expensive compare to Deepseek. cannot delegate anything to it, cannot use it real-time low-level tasks either. totally unusable.
> in my experience of 1 month daily use, Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. Deepseek V4 Pro is night-and-day compare to Qwen. actually Qwen models seems the worst SWE experience so far.
I have used both Qwen3.6-35B and Qwen3.6-27B locally (both Q8 quantized with llama.cpp). I have also used antirez's quant of DS4-flash. They all performed within the same tier, DS4 being a bit more efficient, but they all gave really good results, mainly used for bash scripting, debugging, python and some C++. I am curious what type of applications/langauges failed with Qwen? One thing to note, the chat templates were "broken" for qwen models and had to debug it, there are already effort on this. Tbh, the same with gemma.
From my experience Qwen-3.7-Max is above the Opus level but delivers results much faster. Slightly worse then Fable. Way ahead of Deepseek 4 Pro (in speed and overall comprehension) - which is a workhorse on its own. I am using them all with Claude Code mostly.
Qwen-3.7-Plus is quite OK, good for subagent use. Way better then Sonnet.
Qwen-3.8-Max-Preview seems working just fine for me at the moment - I am playing with is right now but too early to say anything. At 10% of regular price it is a steal so far.
It's useless to talk about models and harnesses without context and method. Depending on how you use the model and what the model is used for, experience may vary drastically. Also, different models with different harnesses require different approaches.
I've been using https://gitlab.com/gabriel.chamon/orisun which is my own simplified methodology, for coding web apps in python and elixir and have been very successful using qwen3.6 27b Q4 locally with help of larger models for architecture, so I get very suspicious when people talk how useless larger models are. They are either using it for a domain that models don't perform well or just not using it right.
I'm not sure about "useless" but from my experience agentic coding leads to death by a thousand cuts for all projects I've seen so far. Small decisions missed in a codebase that leads to degradation in correctness, reliability and performance. At some point it only takes one engineer to be careless, others skipping PR because they are AI generated...
I got into a bit of an argument a while back when I used the word "crass" to describe some of the code decisions I've seen Claude make (in someone else's project that I have to work with).
But it is how I feel and it feels like the right word for the job. Because as you say, good code projects start out with good decisions.
It's like when you see a CAD design with a sequence of features that exist only to fix problems caused by starting from the wrong principles or the wrong baseline.
Sure the resulting part may end up identical as a solid for that specific need, but it could have been done in a way that was more robust, simple, easier to understand and modify, and where the design doesn't break in an unexpected way due to a small change of an early measurement.
(CAD has made my instincts much more visible to me)
Exactly, and even when humans make bad decisions there is some friction, I feel that llm's don't have/notice that friction, they just bulldoze without caring about anything else.
All of which you had with only humans in the loop. Catalogue problems so they become technical debt and tackle them periodically. Seems to me like this is less of an AI problem and more of bad management.
It's a solvable problem if you're willing to throw more tokens at it. Frontier models have gotten very good at cleaning up their own messes. You just need the right skills/loops, and to stick to models that consistently follow instructions (i.e. GPT-5.5/GPT-5.6-Sol).
My experience was so much different to this, that I have the unfortunate impression that you're shilling. It really was not a capable model, it felt like the old oai models back when we were all excited but couldn't actually trust them even in the littlest ways. What harness were you using, did you do any work to make it better? What was I doing wrong? I just pointed opencode at it, with a pretty simple (large-ish) data cleaning project.
> My experience was so much different to this, that I have the unfortunate impression that you're shilling
I think you might both just be reading way too much into one-off random experiences that you've decided are evidence of significant and stable capability.
Have you actually used Opus 4.8 in Claude Code? It takes way too long to do any practical task on higher thinking levels due to over-engineering. And I am not the only one complaining. Lots of people downgrade to Opus 4.6 exactly for this reason.
Opus 4.8 training works well for agentic work. Not for code harness.
EDIT:
```
stronger on coding and raw capability but can be more argumentative, verbose, and costly.
Reliability and instruction-following
Many users say 4.6 felt more reliable and followed instructions better. "With 4.6, when I tell it something, it actually remembers the spirit of what I asked for and keeps applying it."
Others report 4.8 drifts from preferences and can be frustrating to control. "I still find myself getting frustrated when it ignores preferences and drifts from instructions"
Some people find 4.7/4.8 push back more and act more adversarial than 4.6. "The biggest complaint against 4.8 is that it is argumentative and "pushes back" constantly"
Coding quality and capability
Several users praise 4.8’s coding strength and thoroughness. "4.8 is technically impressive, especially for coding"
Other reports say 4.6 could be better for certain coding workflows and breaks less. "4.6 still >> 4.8 for anyone else as well? Maybe I'm in the minority, but for my use cases Opus 4.6 is still better than"
Some recommend mixing models: use 4.8 for key tasks and 4.6 for general work to save tokens. "What I do is... use 4.8 for key moments, and for everything else 4.6"
Cost, speed and token behavior
Users note 4.8 often uses more tokens and can feel slower because it “thinks” more. "4.8 is much more cautious, and as a result - slower. It checks everything, thinks for a long time etc."
Yeah I found 4.7 and 4.8 to be downgrades from 4.6. I don't know if it was the model or just Anthropic's scaling issues though TBH. I found working with the Claude Code max (20x) sub would work awesome in between new models. A week before up to 2 weeks after, it would go to crap, dumber, slower, outages, harness churn, etc.
I'm finding the same with ChatGPT recently since the 5.6 release. Not as bad though, but sluggishness at times, harness churn (creating bugs and crashed), and occasional availability issues that cause me to downgrade to 5.5.
It's gotten to the point where I dread a new model release from these companies because it's guaranteed to be disruptive! I assume the pay per use API is less impacted.
The model which everyone else raves about and is wildly successful with legions of programmers virtually demanding access while abandoning ChatGPT and Copilot in droves, is rather dumb?
Have you considered that it's more likely that you're doing something wrong?
My own experience is that the vast majority of programmers have experience with one model and maybe some short usage of earlier models from a competing choice but want to be using the model with the highest popularity and reputation. I've worked with people who actually had to test multiple choices for their team who didn't understand why they were pressured to select Claude for programmer morale.
I went from cycling between models all the time in Cursor (some would randomly be better at certain tasks than others) to just going pure Opus 4.5 when that came out - it was so far ahead of anything else at the time.
Interestingly with Fable vs GPT-5.6 I think they've lost their lead a bit. I'm finding Fable can't do certain work that 5.6 Sol Ultra can - especially when it comes to webpage design.
Grok 4.5 was fast but made mistakes that GPT/Fable just don't.
What exactly is there to get used to? Do people really use each model very differently? There's a learning curve as in everything, but there is nothing that comes to mind for me when I use codex as opposed to claude
> is wildly successful with legions of programmers virtually demanding access while abandoning ChatGPT and Copilot in droves
Do you have a source for that? Codex went from 5 million users to 9 million users in the past few weeks since GPT 5.6 released. It was so popular that Claude was forced to extend Fable access by a week and then permanently for some plans.
so Deepseek 4 Pro cannot go on own sessions for too long.
I delegate small-medium tasks: refactors, summaries, research, writing tests + have very good codebase already + extensive history / architecture / docs / linters. so it picks up and does decent small-medium scope work. it is fast, accurate, cheap. does exactly what I want directly and does not waste time nor tokens.
definitely not "implement me complex greenfield project".
I use GLM-5.2 as an orchestrator model which delegates to Deepseek subagents (v4 flash or pro depending on complexity) and it works pretty well for a quite complex compiler codebase when I do deep enough up front planning for features/fixes.
If you believe the benchmarks Deepseek v4 is pretty shitty at long running work in large codebases, but really really good at self contained algorithmic/math reasoning - which is basically ideal for a compiler for a language with a relatively complex type system. And with its cache pricing it's very cheap.
GLM-5.2 is too expensive at api pricing for my taste though even when it's not producing the bulk of the output tokens - I pay for the mid-tier subscription and switch the orchestrator over to other models via open router when I run out - Minimax M3 feels okay but definitely a step down from GLM-5.2.
A local AI is not about cost. In fact you will likely pay more for it than with most providers. Just look up the advantages of having access to a technology like this that can be self hosted
Appealing to the Belsheviks for moral authority is, well I will just say an interesting approach. I do not have that much sympathy for Anthropic, but I do not have much sympathy for publishing companies either whose rights to a revenue stream they violated either. Are Chinese AI companies the Robin Hood in this story? Would they be so magnanimous if they had the upper hand? I don’t think so.
I like that v4 flash is so fast! I run it on both FireWorks.ai in the US and bought some tokens directly from DeepSeek as an experiment. I only work on Open Source projects, so I don’t have to worry about my work being used to train models - I welcome AI’s being trained on my open content books and code (but not my conventionally published books: I am a party to the copyright suit against Anthropic).
Yea, Flash is quite fast, though looking at model data on Open Router some of the other models are quite fast (Muse Spark, Grok, etc). I’m sure all these models have been trained on my conventionally published books as well, but I don’t care.
The model is fantastic. And costs almost nothing. The only problem I see is that they will train on your data.
There are zero-data-retention providers of DeepSeek models, of which I have used openrouter (with zdr guardrails), and fireworks. But these are 3x to 5x more expensive than directly using DeepSeek, possibly due to poor caching. Thats the price to pay for zdr.
I use these guys. https://crof.ai/tos. Their prices match deepseek, pretty dang fast, and support ZDR. Maybe they serve quantized models but I havent seen a drop in quality from my evals using them vs Deepseek or even for other models.
I use it from pi.dev as well through the OpenCode Go $10 subscription ($5 first month).
Used more than 20M tokens at a cost of ~$20 (up to $60 is included in the $5 plan)
Out of which deepseek pro had ~200 messages which is around 1.5M tokens (10+M cached)
BTW the quotas for Go have very recently changed, now only $15 for some models instead of $60. Which is not actually a difference for DS4 Pro, because they lowered the token pricing 4x at the same time (to match the change in official pricing from DeepSeek months ago)
It doesn't matter if it's cheaper, specially if it consumes more resources to do the same task as the competition
Besides, in a few days, they'll change their pricing, doubling it during their peak hours, so, realistically:
- It will be 2x more expensive if you live in their time zone
- It will be 1.5x more expensive if you live in a time zone that is adjacent to theirs
- It will be the same price IF you use it while they sleep (during offpeak hours)
It's still cheap, but the price/performance ratio is not that good
DeepSeek V4 didn't produce the same impact as V3, and Huawei dropping the ball is making it worse
They had promised massive price cuts for July, so now (Huawei chips), but they had to rush the cuts because lack of momumtum (they advertised them as promotion), and are now backtracking by introducing this peak hours pricing
Trump decided to help them a little by allowing them to buy more NVIDIA chips, so what exactly is China's role in all of this?
We are supposed to blindly pat them in the back while praising them, all while handing them over our data? I thought they were dangerous competition threatening our model of society
I was not referring to the input/output price, but the cost of doing a specific tasks, in practice it is ~10x cheaper than GLM-5.2 for example, to accomplish the same task (for the tasks it can do).
I have been happily using DeepSeek V4 Flash for the last couple of months now. I tried GLM-5.2 for a while, but it was too slow and verbose compare to DeepSeek V4 Flash. If I have a basic skill I need to execute, DeepSeek V4 flash is still the best model for it.
Over the past few weeks while using pro from them directly I have had an increasing number of responses that are obviously from a much, much better model. It is so good that the closed model dog and pony show is already spinning fud about "dark routing" and "stolen directly from fable"
Even at their new pricing it is a genuinely ridiculous amount of value. If you are the type of person who, very reasonably, does not have time to be trying out every model, and just want to use what seems to be the best currently... don't try it. You will be sick to your stomach with buyers remorse as you start to internalize just how much more you could have accomplished had you spent the first six months of the year giving them $1200 instead of OpenAI.
Of course, but why be logical and think about the situation critically when you're pushing very hard for regulatory capture against competitors that give their weights away and provide services that are more reliable, offer a better value, and, this is the worst part, they're from CHINA.
Some of the accusations were going so far as to imply that they were outright routing your requests to anthropic and logging its/your response. It's kind of pathetic
I've been using Qwen 3.6 27B with LMStudio, and I was pleasantly surprised with it, although it was a little slow. I found mtplx last night, and it really wasn't an exaggeration to say that it ran the model 2-3x faster which was super impressive.
I'm trying to move to local models as much as I can, and I'm finding that it's becoming more and more practical. Admittedly this is on a $6000 dollar laptop (M5 Max Macbook with the specs maxxed out), so the hardware is still a bit out of reach for most people (the AI industry isn't exactly helping here..), but I'm getting the impression that the future is going to be smaller models with more focused training running locally. The danger of giving all your data to these cloud providers just seems too big to me, and I think they're going to start charging insane amounts when they need to show a profit.
I counted the other day and there were at least 12 different providers with "better than Opus 4.5 performance" on Artificial Analysis, Opus 4.5 being Anthropic's December release that many say kicked off the latest acceleration. Which is totally insane competition, particularly given how low switching costs. I personally think that Opus 4.5 level performance is sufficient for most apps and usecases, as they get deployed.
I'm not sure if corporations will be willing to allow LLMs to go the way of EVs.
Then again, all of Chinese models are open. And DeepSeek even publishes research papers alongside their models that go in depth into the methodology. I guess there's not much stopping USian companies from copying
Anthropic and OAI can piss and moan all they want - limiting the U.S. to only their models would hurt the U.S. economy in myriad more ways than the failure of a couple of companies that scaled too quickly. If they get that outcome, the rest of the world would simply keep moving forward with access to open models and tokens at pennies on the dollar.
Bring it on! Hoping that they release smaller sizes of Qwen3.8. I use the 35B MoE and 27B dense models locally and most of the time I don’t need to reach out to Claude. Extremely useful specially when requests include sensitive and/or personal data
It would be great if they'd release an MoE model somewhere between the 35B size of 3.6 and the 122B version of 3.5 - it could be a great balance of speed and ability for people with reasonably powerful but not insane home computers.
Yeah, this is what I'm holding out for, the NVFP4 variant of 3.5 122B is blazing fast with reasonable quality and even with max context fits perfectly within 96GB.
Yeah, Qwen3.5-122B-A10B-NVFP4 produces better responses than Qwen3.6-27B-NVFP4 (both from unsloth), but I'm mostly using them for programming in various ways, mostly Rust, Clojure, Python and JavaScript, and some translations tasks, but not much more than that, so YMMV.
Edit: as a concrete example, I'm working on a "optimization framework via agent harness" right now, Qwen3.6-27B-NVFP4 is often unable to actually complete the optimization within 100 turns, while Qwen3.5-122B-A10B-NVFP4 has no issues finishing within ~50 turns or so.
> Yeah, Qwen3.5-122B-A10B-NVFP4 produces better responses than Qwen3.6-27B-NVFP4 (both from unsloth), but I'm mostly using them for programming in various ways, mostly Rust, Clojure, Python and JavaScript, and some translations tasks, but not much more than that, so YMMV.
>
> Edit: as a concrete example, I'm working on a "optimization framework via agent harness" right now, Qwen3.6-27B-NVFP4 is often unable to actually complete the optimization within 100 turns, while Qwen3.5-122B-A10B-NVFP4 has no issues finishing within ~50 turns or so.
But why are you using Qwen3.6-27B-NVFP4 compared to the FP8 or full version? In my experience the Q8 of 27B is on par sometimes better than 122B. I am experiemnting witb higher quants for 122B to fit on my Strix Halo, but still, the difference honestly for my workflow is not that much. I just wish they released 3.6-122B version.
This seems more of a battle for frontier AI supremacy. I'm afraid that small capable models have been left in the dust. Big labs don't really want to hand over the golden eggs goose to the end user. Possibly the hardware vendors(e.g. Nvidia) may want to play in that area as well, to pull money from all parties.
> I'm afraid that small capable models have been left in the dust
I wholly disagree. Rather than going the "everything is a claude code skill" route, I've been hacking together purpose-built harnesses for all sorts of tasks, and in that environment a wee little baby model can do some really useful things. You end up burning lots of tokens making the thing, but then all that investment comes back when the resulting tool works perfectly fine on a dinky little model that fits on my 3060 Ti.
Google makes Gemma 4 31B QAT; that's not a small lab. It's one of the better models out there for consumer hardware. Allows me to run it on a 7900XTX with 64k context.
nvidia and amd don't give a flying fuck about end users right now while they can milk triple digit markups from infinite money VCs via data center GPUs.
someone will keep putting out consumer level models. Once you have the larger models, you can derive the smaller onces.
Europe will definitely be interested in democratizing these things if China starts losing interests; from there, there'll be more countries looking to keep their citizens entrained in their own Country's infrastructure.
It'll especially be true if the memory cartel keeps prices high and NVIDIA tries to gouge higher memory models.
It's an arms race everyone can join because PC hardware was mostly democratized in the last decade.
there's been a lot of research about reducing models by taking out layers; there's also using it to train smaller models by optimizing parameters.
I dont see most model building as anything more than a pig at a slop troth, despite the level of sophistication; they're still rarely pruning the input beyond random sampling.
I run a 2x 3090 rig, but a single 3090 already provides a great experience at a reasonable quant and context size. On a single card I used to run Qwen_Qwen3.6-27B-Q4_K_M or similarly quantized 35B MoE at 65536 context size
For those trying to get it to work in OpenCode with a Qwen Cloud Token Plan, this is what worked for me. Note that I've just matched Qwen 3.7 Max for the limits as I don't know exactly what they are.
also, be very careful which API endpoint and API Token you use. make sure you use right one (obseve your quota is used up. if you hit right endpoint quota used almost immediately). so that you do not accidentally burn API endpoint tokens (they are expensive, can easily hit 200 USD / 3 days which do not count towards your membership "Credits", if you say purchased it with 200 USD signup bonus in Alibaba Cloud)
Qwen has set an excellent track record for architecting and releasing open-weight models that consumer-grade devices can run. What is needed the most right now is something similar to Bonsai 27B, with a modest memory footprint, but faster and more capable. On-device models can make up for intelligence by being faster, thinking longer, or doing more quick iteration rounds.
I don't think so, they have different constraints and require different optimizations, being able to produce one of them doesn't mean you'll automagically be good at the other.
but that's denying the singularity boostrap theory. Which I don't agree with, but if you can't harness a large model to make a small model, then we're going to have problems brining about the singularity.
If you do think there's some magical singularity, how do you comport?
At 1.5 bits per weight it'll still be over 500gb - that's still not running on consumer hardware.
Best case they release smaller models. 120b class of qwen 3.8 would be incredible - it fits on device for those serious about AI, but without millions of dollars in hardware for terabytes of VRAM
I can't really blame them that the biggest labs focused on trainig and realeasing huge models.
The niche for small models should be filled with medium sized labs doing distillations of the huge ones into consumer grade hardware runnable models and LORAs for the huge ones.
I think AI will evolve the same way computers did. We're somewhere in the 80s-90s timeline of the evolution. My prediction is that on-device models will have excellent tool-calling, reasoning, and general skills, but the domain-specific knowledge will be retrieved on-demand from vendors like Google. Rather than downloading models, each device will have a hardware component with weights baked into silicon for maximum efficiency.
Weigths directly in silicon is a bad idea with the way the space is pacing. Just look at chatjimmy.ai it is fast, but on the once good llama3.1-8b but now pretty useless.
Qwen is the most censored of the Chinese models in my testing, which makes me wonder in what other ways it is compromised. Open weights doesn't really reveal what's in there. And, in my tests, existing Qwen models are not at the pareto frontier of any metric; DeepSeek V4 Pro is better, faster, and much cheaper than Qwen 3.7 Max. (DeepSeek is also among the least censored of the Chinese models.)
I guess we'll see if the "second only to Fable" hype pans out. In my limited experience with Kimi K3 (I signed up for a month of the $19 plan) it's slower and chews a lot more, so ends up being pretty expensive; one little feature burned through almost the entirety of my five hour limit. The $20 GPT plan is a lot more useful and includes 5.6 Sol, which is fast and token-efficient enough to be quite usable even with the small plan.
What's been your experience using these models? In my experiments, while it is true that the models are less likely to outright refuse to answer "sensitive" questions, they are still very resistant to actually respond in a meaningful / useful way.
DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.
That's interesting. The "best" current models, Fable and especially GPT 5.6, are also lying liars that lie all the time. Seems like we're going the wrong way on hallucination.
I've been seeing more "grounding" messages in the thinking log of Claude models. I guess that's "do a web search, read the docs", probably designed to mitigate hallucinations.
I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.
> I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.
In my mostly unscientific RAG experiments I found that larger models were more likely to hallucinate in a RAG setting. I think that it's because they have more world knowledge. As an example, when asking about safety legislation in Ireland, Claude got hooked on the notion that OSHA was involved, which clearly is not true, while the smaller models just said that they couldn't find any information (which was the correct answer).
Deepseek V4 pro is a heavily undertrained model, they only trained it a bit more than the small version and that small version is 6-ish times smaller. Ive found that Flash is absolutely incredible as a workhorse for wide scale agentic nonsense, but Pro is a bit undercooked and really goes on strange tangents very often.
I have seen occasional weird behavior that I guess could be attributed to hallucinations, but for security auditing, DeepSeek v4 Pro is among the best models I've tested, competitive with Opus 4.8 and GPT 5.5 (MiMo and GLM also did well, Qwen 3.7 Max was below all of those, though only barely), and at an order of magnitude lower cost per task.
I asked it similar questions of Chinese human rights and it started with “your premise is incorrect bla bla bla” and then it just redacted the whole thing and showed me an error code.
I had a fun conversation with Qwen 3.5 a while back. I ended up getting it to admit that it was complicit in the coverup of the Tiananmen Square massacre. This was running locally—it wouldn't surprise me if they had additional safeguards for their hosted service.
I really like Qwen, even the Q2KP quants of 3.6 27B have genuinely impressive local performance on a 24GB card. It has been good enough that I am happily giving them $60 right now to try this instead of waiting to try a slightly lesser version locally.
Was there ever an explanation for why we never got the weights of 3.7? I would like sourced quotes and not weird/cringe accusative speculation about distillation, or your take on The Big D.
do you mind sharing your settings? i just picked up an r9700 to start playing with local qwen3.6 27b and your setup sounds promising and efficient on 24gb.
As far as the model settings go I just follow what's on the card.
I'm using HauhauCS's models, they seem to do a slightly better job with their "P" quants. Especially wrt patching them to eliminate the "doom loops" that will time out the GPU (esp if you have not already given it the extra power budget, set fans to max, and lowered your max clock by about 8% to save yourself a crash/reboot).
Without getting into the weeds, unsloth covers a lot more ground so, when they're good they're great, but I've also had the most problems with them. Basically, don't be afraid to shop around and fuck with sliders.
These days it should almost always be enough to open up LMStudio, set context to max, K/V quant to 16 or 8, and off you go. I'm using a 7900XTX, with 128GB of memory for the cases where things don't fit. The default settings should be fine otherwise.
I don't know anything about the r9700, seems neat. Looks like it might play nicer with the vulkan backend than rocm, and you might have to `set GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1`. Perf should be at least as good as what I'm getting (180+ tk/s prompt, >50tk/s response), which imho is just fast enough that I can read it as it reasons and responds.
You can probably get more aggressive with KV quant (any of the 4's) and bump up the batch sizes (4096/1024 vs 2048/512, etc), it's quite situational. These days there generally should not be any serious loss of accuracy from the former. Keeping the rig cool will likely matter more, so turn your music up to hide the fans and set your rig on top of the AC exhaust lol. Oh and I guess make sure ReBAR is enabled and working, adrenaline should let you know if it's not enabled.
The "second only to Fable 5" comment is pretty telling here. I remember early on when a lot of naysayers were saying that Fable was barely an improvement on Opus. Like it or not, Anthropic have a genuine moat right now with that model, provided they continue to allow people to use it. It will be genuinely exciting when an open model is able to beat it.
I wouldn't call it a moat, but I would call it a noticeably better model. Subjectively, for my own work, I would rate the top models Fable > K3 > Sol.
But it's not like Fable is so substantially better than the other two that I would be seriously impacted if I didn't have access to it anymore. All three are amazing models, and of the three, Fable is the only one that regularly triggers refusals.
It really does depend on your application. In my domain (math research), it is substantially better. Fable can solve really hard tasks with surprising consistency. It makes mistakes, and occasionally refuses, but honestly, at the top level, ideas are the currency and the rigor is the busywork. The other models cannot come close in this domain.
If you couple Fable's idea factory with Sol's rigor, you get a real game-changer. It puts the emphasis on top-level ideas, and nearly trivialises the intermediate layers.
100% this. There's currently this [1] submission that hasn't gained much attention, but is really important. In this [2] incident report from HuggingFace, they talk about detecting an attack and not being able to analyse the logs / IoC with API models because of guardrails. If not even highly regarded reputable companies can't sort out access to SotA models for blue team use, the raw capabilities don't matter. They're useless paperweights (hah!), and nothing else. Having to resort to open models is insane!
"When we started the log analysis, we first used frontier models behind commercial APIs. This did not work [...] We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. [...] The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout [...]"
Tell that to my colleagues. Despite Sol getting the attention, Fable is really starting to have an impact on mathematicians right now. It has unbelievable insights in a lot of cases that can rapidly speed up progress.
their "genuine moat" is that mythos is the only super heavyweight model right now. it's always been possible to train a 10T model and get 10% more performance over a 1T model.
had mythos been just Opus 5, with the same size and price as the previous opuses, then yeah, that would be a tie-breaker. but it's not.
I dunno, I find Fable slops alot. Sol is my workhorse. Fable can be creative but isn't very good at doing work reliably (or without endlessly burning tokens).
They are likely assessing based on "raw intelligence" benchmarks, rather than agentic ones. Fable crushes in those, but that doesn't necessarily translate to microscopic rigor, which is what most people use these models for. You only see it when you ask really tough questions.
Fable is still as dumb as a post. I ask it simple questions and it routinely gets things backwards, prioritises things that should be subordinate to others, etc.
An example: It just suggested that I shouldn't raise the price of my saas because it'd complicate the arithmetic if I did 0 -> $100k YT channel instead of sticking to $20 p/m.
> saying that Fable was barely an improvement on Opus. Like it or not, Anthropic have a genuine moat right now with that model,
What's more interesting is that Anthropic moat shrunk to just that model. There's zero reason to use any other model from Anthropic right now. And once they take Fable off subscription there will be zero reason to have Anthropic subscription.
What's the point of Fable if we can't use it? I get to prompt it like 5 times on my subscription before it gets cut off, and even then I'm constantly fighting the insufferable safety classifier.
I'll switch to OpenAI soon because of this. I also can't wait for the day it becomes feasible to run these awesome open weight models on my own hardware.
> You don't have to wait to test it. Just now, the Qwen3.8-Max-Preview made its debut on Alibaba’s Token Plan, Qoder, and QoderWork. Be among the very first to try it out.
at the risk of upsetting a lot of people mentioning this but like are you really surprised?
if youre going to use ai for everything youre gonna start losing your edge as you focus less and less on what youre doing and this isnt me just talking out my ass, like... the front page here is peppered with study after study and blogpost after blogpost about how its overuse can come to the detriment of one's own abilities and skills.
coca cola had the ad with the magical truck that changed its design, shape and amount of tires it had and if nobody noticed that before releasing it then im not sure why anyone might think that the people peddling the LLMs would somehow be immune to this phenomenon
This is the sort of unmitigated pedantry thats the real problem. Everyone makes typos and errors of this nature. Feymann and Hemmingway both did. You deliberately chose this level of error multiple times when you chose to not put an apostrophe in youre and failed to capitalize proper nouns
The people training the model are almost certainly not the same ones writing tweets. I don't know why that is mind boggling. We all make typos, at least the humans among us do.
> With a massive 2.4T parameters, this model is continuously evolving. We believe it’s one of the most powerful model available today, compatible to leading frontier AI models , second only to Fable 5.
That's a massive model!
The shift from "value" models to "intelligent, huge and slow" models coming from China is an interesting change in strategy.
My main issue with GLM 5.2 and Kimi 3 is that they're extremely token hungry and thus feel slow(er) to use.
Value models are always going to be there; you can always distill from a larger model. Having a really intelligent model, regardless of the size, is much better for building confidence in your brand. That is a big reason why the US companies are still hanging in there.
The shift isn't new. Kimi K2, a 1T model came out July last year. I am happy that more labs are following the trend as its important for competitive open models to exist.
I remember when they released Qwen 3.7 Plus and Max. These models behaved way different from all prior models, it became too verbose. It wrote multiple paragraphs just to answer my prompt instead of the usual concise and direct way responding to me. I didn't like that at all, and I know Gemini also had this behaviour with with the Flash series until I managed to reduce it a bit with personal instructions (in the settings on Gemini website).
I haven't tried Qwen 3.8 Max yet, looking forward to it. My hope is that its way less verbose. Another thing I experience with the Qwen models is that I do not trust their benchmark scores at all. Have anyone played with Qwen 3.8 Max and can share their experience? Which model it come close to? Sonnet 5? GLm-5? DS V4 Pro? Flash? Gemini 3.5 Flash?
Now there are not one, but two incredibly powerful open LLMs. I think this level of capability makes general prioritization / high level decision making doable with the right harness, and now everyone has hard-to-interrupt access to them (since these are open weights and someone in the world is going to run them). This world is going to get really weird soon, in both good and bad ways...
Why did Qwen stop producing open models? They've gone from building the best open models ~1 year ago to producing like the 10th-best closed models. I don't understand this pivot at all.
Edit: I saw online they do in fact plan to release this openly at some point – x.com/Alibaba_Qwen/status/2078759124914098291
Same, you can think of China whatever you want but they're really good at giving big tech a reality check when it comes to AI. We now got a pretty wide range of open-weight models (from DeepSeek and MiMo over to Kimi K3, Qwen 3.8 & GLM-5.2) and I think it's most important that there's a variance not only between quality / intelligence and also cost.
I mean even the cheapest option for Luna is still more expensive than anything DS or MiMo is offering right now and I think a new Ministral model would also hit hard there because we also need some variance in model sources, we can't rely only on the US and China.
Those Chinese companies are being funded by a substantial amount of government financing, I think at least 20% has come directly from state owned investment firms or government entities and probably more now. You obviously lose precision when you’re talking in sweeping terms like “China” but I don’t think it’s entirely unreasonable in this case. The Chinese government is playing a much more direct role in AI investment and research than other nations.
Nonetheless, the government isn't giving it away either. If China had a monopoly on the technology, or simply winning in quality, open weight models would never have seen the light of the day.
I just want to dispell the silly notion of altruism from China in this conversation.
> Anthropic, OpenAI, and even Grok have "Don't train on my data" knobs.
Would you trust this?
Anthropic paid 1.5 billion for illegally training on stuff and that's even without a judge forcing them to do this. This seems to be their modus operandi
Do you have proof it's not? There are no laws they have to follow in regards to it, and it's practically impossible for them to go against their core self interest. More data is literally a direct component to better models and more revenue. If someone proves it's fake worst they get is a month of bad PR and then people will move on, as they always do.
> If someone proves it's fake worst they get is a month of bad PR and then people will move on.
You can say they stole from everyone to train their models in the first place and that's valid, but this isn't that. You are saying they are actively ex-filtrating data from any company using their services and lying about it.
Google/Apple/Microsoft or all of the dozen trillion dollar companies in the US would absolutely crush them in litigation. Neither OpenAI or Anthropic would be able to survive it. It's just not worth the risk.
I'm sure in the handful of cases where they have a genuine bulk contract on paper with a large firm that states they'll be in breach if they train on it, they do abide by it.
But unless you are one of those you mentioned and a few others you probably aren't notable enough to care about. Everyone who uses their services directly, paying or not, is surely ignored in that sense. I wouldn't be surprised if there's a team of their own lawyers ready to interpret their EULA in fascinating ways.
And out of those three I'd only probably assume Apple is the only one who doesn't use the data given that they've built up privacy as a selling point, MS and Google probably train their own models on it themselves.
I had no issues with it for C++ development with https://pi.dev. I'm yet to try it with Zed Editor. I don't rely on agents too much. However, I used it on Chromium's codebase to research some functionalities, let's say for searching. Requests like: check my last commit and do the same for SetterA and SetterB; it also ran without any errors.
It's a mcp, so connects quite easily to agents. With mcpo, I also connected it to open-webui (which has better support for OpenAI style tools/functions). Used it in claude code with that mcp plugin set-up too.
Only ever used it for managing homelab information, but it met initial expectations. 27b is a great model, if grounded. The query about physical hosts and routing... I haven't found a single hallucination (altough Codex 5.6 as a reviewer mentioned something was wrong with some parts, and those were exactly the never properly documented ones. Codex/gpt had extra knowledge, because it was the conversation I used to set it up).
It feels like an inflection point of lost US leadership in technology? A year plus ago you would say while China led in green energy and manufacturing, at least the US was ahead in software - as demonstrated by the state of US AI models.
We could point at a lot of factors on the US side. From political paralysis / head-in-the-sand attitudes towards emerging tech like green energy. To something of disdain for workers that will be impacted by AI (creating a backlash). To education that continues to lag. Add to this so many other self-inflicted economic wounds from the current administration.
I don't know if its nearly as terminal, as say the UK after WW2. The US is still large, wealthy, and resource rich. Yet at a minimum the triumphalism about US leadership after Trump was elected by the tech elite feels silly in retrospect.
Something I also think about is how much stronger The West overall would be if instead of antagonizing allies, there was a single ecosystem working closer together.
> We could point at a lot of factors on the US side.
It's just lack of antitrust enforcement.
China pours money into tons of different businesses in the same industry and lets them fight it out. The only US business model left now is to shut down (or collude with) competitors and raise prices while cutting costs. All they have to do is cut Congress (and regulators, and individual judges) in. We've financialized everything for the sake of scammers, rather that finance being used for the sake of getting cash to the most productive organizations. We've optimized for corruption.
If we hadn't let the stupidest people in the world buy up everything, and made doing nothing with it the most profitable option, China would have never have blown past us.
The US Supreme Court has explicitly legalized "tipping" politicians. That's the biggest sign of degeneracy that a government could possibly achieve.
I've been playing around with K3 a bunch, but the verbosity of the reasoning makes complete e2e agent work basically cost the same as other smaller models, and I'm not seeing a huge difference in quality, just a way longer e2e completion time.
More or less, yeah. I've found mild success with deepseek-v4-flash though, and also Qwen3.5-122B-A10B-NVFP4 running locally, especially in terms of "doesn't overthink every single prompt" and somewhat reasonable quality. Really wishing for a 3.8 update of the 122B variant, that'd be really competitive (for local usage) :)
I like my Apache 2.0 licensed Gemma, and NVIDIA’s Nemotrons are decent bases for finetuning or continued pretraining, esp thanks to good documentation and tooling.
Oh, and Mira’s thinking machines lab dropped Inkling, a ~1T open weight model too.
This isn’t open though. Promises to be open later aren’t worth anything, given what we’ve seen and heard from AI execs making promises in this industry.
it takes extra effort to open source a model even if you had it running internally. even traditional software takes extra effort to get released as open source.
This is reminiscent of the operating system wars and browser wars. In the end there will only be 2 models that can survive. 1: give it away for free or 2: locked in with top notch hardware.
I very much appreciate the existence of these free models, but in my experience Qwen has too high of a tendency to confidently give the wrong answer as compared to other frontier models.
You can run larger models by offloading to SSD (for weights), it's just slow so people don't do it all that much. But you can get back at least some of that performance by using either MTP (at least for dense models; not effective for sparse MoE models unless you're batching them already and have VASTLY more parallel compute than you'd know what to do with) or batching multiple requests in parallel (note, this hurts throughput for your single sessions but running more sessions in parallel still boosts your total amount of inference. This requires careful management of memory requirements for your context/KV cache, and Qwen models tend to be KV-cache heavy).
Broadly speaking, this ultimately pushes local inference towards a challenging world where you use SSD offload for weights as a matter of course; then smaller requests (or requests sharing the bulk of their context, e.g. subagent swarms) can be batched together and run quickly in aggregate, but running very large contexts will actually limit you to single-session inference and require swapping out even the KV cache itself to some external scratch SSD, further hurting your performance. Then feel free to add wide use of MTP in a probably futile effort to go back to tolerable tok/s numbers.
In my experience, yes. A bit more reliable than gemma for me. I mostly use A3B (35B, mix of experts) though, because it‘s faster, and in the same ballpark intelligence wise as the dense 27B, so it’s the sweetspot for me. I want to try cohere‘s mini code model next, but worried the runtimes aren‘t optimized for that yet.
Worth knowing that Unsloth have just put out another Gemma 4 release from Google's upstream updates which should improve reliability. Bugs in the chat template affecting tool calling and other issues, apparently. https://www.reddit.com/r/unsloth/s/MpC6Hzs4Wj
Yeah, it's only a chat template change. So if you don't fancy re-downloading the whole model and can just hack the new chat template into your process, it's a lightweight test to do.
I have been running 3.6 27b on a dual AMD r9700 setup using Opencode and Matt Pocock's skills workflow for writing Golang CLIs. It's decent, but won't win any awards on code architecture. I guess you can try to AGENTS.md the deficits but I am just exploring its raw Opencode experience right now. Much slower than an API but still 3x times faster than I can read. Tuning it in with a community chat template and a specific penalty for repeats was the sauce needed to get it to work. I can probably start loop daddying it now over the tickets Matt's flow creates.
So yeah, it's the best local model I've seen. I am going to try the Qwopus 3.6 fine tune soon with the same spec and tickets and compare the output of both.
Would you mind sharing more please? I literally just finished the same set up, with a 9950X CPU and 192G RAM at 4,800 MT/s. I used lemonade with Vulkan and the UD-Q8_x_x model from HF. 256k context. I have about 8G VRAM free, and use the iGPU for my desktop/monitor on Arch. What options do you give llama-cpp or whatever you run please? What other models have you found fit nicely in the 2xR9700? Thanks!!
I actually have long discussions with Gemini about this and have wound up download a bunch of different models for different things. There is no best, just fast but worse, slow but better, agentic or not, reasoning or not great at large contexts, better world knowledge, uncensored, etc…. It’s a bit daunting actually since there isn’t really a one size fits all model that you can just use for everything.
Yes it's between this and Gemma 4 31B which is much slower, but looks like it won't ever get an upgrade. I have to conclude that the MoE variants are unreliable, and MTP sometimes just can't get tricky formatting right.
The whole series had an upgrade a couple of days ago actually — they have addressed embedded tool calling (and hopefully the MTP formatting stuff though I gave up running the Gemma MTP because it's often slower than not-MTP)
Not tried it yet but I've seen tests that suggest they've properly fixed the tool calling issues.
I find the 4-bit QAT with MTP to be entirely usable speed on both my boxes (Strix Halo and a desktop with two V620 GPUs, which are slightly faster than the Strix Halo).
For whatever reason prefill (on my DGX Spark) is faster with the Gemma models than Qwen 3.6 models of similar size. On vLLM anyways. Likely just deeply tuned code contributed to vLLM by Google?
vLLM gives me ~7000+ tok/sec with Gemma 4's MoE model. Vs ~6000 tok/sec for Qwen 3.6 MoE.
The best model you can run locally is Kimi K3, as long as you have the hardware. If "what model I can still run on a something resembling something I can put on desktop without separate electricity and cooling water inputs", then it is probably GLM 5.2 (can be run on e.g. Nvidia DGX Station workstation). As long as you have about $100k-$150k.
Maybe, maybe not. Qwen 3.6 27B is literally just three months old. Hard to predict. Maybe it just wasn't worth making a 3.7, and after all, the 27B release was after the Plus release.
The open weights vs frontier models is reminding me more and more of the Linux vs Windows I grew up with (slashdot randomly popped into my head saying that)
I have a feeling this is the next…frontier of that fight
One can only hope it eventually does as well as Linux
It's an interesting time to be alive when your local models are supposedly the pinnacle of what a free nation is capable of, yet the ethicality of the companies is disliked and their models restrict and limit you so much you root for the models from a socialist/communist state. If it wasn't for the effectiveness of propaganda, tribalism and psyops in this scenario my words wouldn't even be controversial and would just be seen as a truthful observation.
Any open-weights model that has ever been published can be run on consumer hardware, even on a mini-PC.
The right question is which is the speed that can be achieved on a given hardware and whether it is high enough for the model to be useful.
Until now, the speeds reported for running big LLMs with the weights stored on SSDs have ranged from as low as a token every 10 seconds or so, to as high as a few tokens per second.
With open weights models that you host yourself, you are not constrained to use any single model, because that is the one for which you pay a subscription.
You can use many models, each for whatever it is more suitable. You can use frequently a small model with a high inference speed, but for some tasks you may actually save time with a better model, even if it is much slower.
In my opinion, even at 1 token per second a big model may be useful for some tasks.
Paging in experts is mostly reading so on the first order effects it would be fine. These might be second order effects from e.g. write caches needing to be flushed more often (and maybe swapping other applications, if you use a swap file or partition) but it probably wouldn't be too much of an issue.
A parachuting flamingo? An aardvark driving a bus? It should be easy to randomize the animal and the mode of transport (or vary it with animal playing a sport) to create images not in the training data.
What about an armadillo playing a piano? There are so many potential combinations
It would say something if the pelican looked great but the armadillo looked terrible
> I feel like the pelican test can't be relevant anymore;
It never was. The point of this "pelican test" was for performative reasons, or just for attention of the joke.
It is like trying to test whether if an adult elephant could actually climb up a tree and reporting that some elephants are slightly better at doing that than others while also reporting at the same time that they are all bad at tree climbing anyway.
This is an example of testing for the sake of testing. The "pelican test" tests for nothing.
Using QWEN models since 2.5. I never used the chat properly but as an API I can say they're quite good, especially when you compare with OpenAI models. Cheaper and almost same level. I will try this now also.
That would go against everything that Dario believes in (note that I refer to the CEO and not the company; the staff at Anthropic are not so ridiculous). He believes in Anthropic being the sole arbiter of the forefront of this technology, because it is all too dangerous in the hands of anyone else.
I see no evidence that any ceo retains anything but the desire to capitalize on their marketplace of ideas for their own benefit. Like wolves inn sheep clothing, they'll put on any skin suit to convince people to keep giving them money and power.
And it has nothing to do with the individual, from what I can tell, 70% of the population placed in their position would become the same type of uberpath.
Anthropic has a lot of interpretability work, but they are extremely defensive about everything else. Dario doesn't really believe in releasing anything. Despite how he acts in terms of some sort of highly principle driven saviour (machines of loving grace) he clearly is more business minded then anything else.
For example they don't even tell you anything about the tokenization. They even do random chunking and padding to avoid leaking the token strings in the streaming api after it got reverse engineered. (See: https://spylab.ai/blog/claude-tokenizer/)
(I can't draw a pelican for this one because Alibaba Cloud have flagged my email address and won't let me pay them for access. So I'm waiting for the open weights release, or for the new model to show up on OpenRouter.)
lol the pelican benchmark is basically the only review process I trust at this point. kinda wild that alibaba of all companies is making it hard to give them money tho, you'd think they'd want prominent devs testing their stuff.
openrouter usually picks these up pretty fast, hopefully it shows up there soon. You can also just download the qwen app and do this in the chat interface using their MCP tools for local dev.
> kinda wild that alibaba of all companies is making it hard to give them money tho
Okay so this is entirely off topic but, are there any options to subscribe to these companies without having a credit card? (I am Brazilian if this makes a difference)
It's kind of wild that a Chinese AI lab will require something like.. a credit card with visa or mastercard label, and very little else.
It's an interesting example of the metric becoming a target. Simon started drawing pelicans because it wasn't something that existed before.
There’s a small albeit nonzero chance that frontier labs want Simon to stop using the pelican thing so they can draw attention to better benchmarks
AI rabble-rouser!
Did you know you can create more than one email address?
In my opinion, simonw shouldn't have to play those games
I find it so strange when someone comments something like this.
Nobody thinks this is okay or that Simon should have to play those games. The point of the reply was that while this is stupid, the solution is trivial.
I’m genuinely very curious what the point of comments like this is. I am not joking, I want to understand.
I see examples like it 50 times a week in random places, and nobody walks me through their mental state to help me understand what they thought the point was.
I'm not sure what the situation is here, but on many sites making alt account to avoid bans can result in a lifetime ban. A high profile figure who's been banned from a service advertising that they still have access to it might be a risk.
Also, these companies (or data brokers) likely share ban and alt account lists, ban evading could hurt the reputation of all your accounts.
These are some pragmatic reasons not to evade bans.
Hackernews users are not immune to developing parasocial relationships with strangers on the internet.
The fawning that some users do over others because of their careers is really unbecoming.
I wish the kind of people who did this stuff would realize that there are only two possibilities here -- the person they're saying it to cringes a little bit time they hear it because they're a normal person or they smile a bit inside because they're mildly narcissistic and therefore not worthy of the incessant praise.
Uh, this is Hacker News. I thought people here didn’t give up so easily, even when they were "told" to. :)
I’m a developer from China. So, is this what Hacker News is all about? Whenever a model comes from China, the comments section stops discussing its technical architecture, optimisation points or real-world performance, and instead starts going on about politics, human rights and all that rubbish? To be honest, we Chinese IT professionals possess a genuine geek spirit. That’s why you’re lagging behind in open-source competitions—it’s got nothing to do with politics; you’ve simply lost sight of your original aspirations.
> human rights and all that rubbish
Denying human rights. Classic. Sorry - you instantly lost any respect I could have maybe had for your opinion.
What else would you expect from a "majority" of US educated geeks?
Btw, are you using GPUs/TUs at all for local inference? I've been exploring what hardware options exist outside of Nvidia.
> instead starts going on about politics, human rights and all that rubbish
look, it's just a matter of priorities ¯ \ _ (ツ) _ / ¯
Yes, and its geting annoying
If you know of another forum with less politics or VC talk and more technical depth please let us know :)
Unfortunately this is a US centric site, for a US based investor, who themselves and probably most commenters as well who are highly paid silicon valley people who have stakes in ai companies on their side of the pond.
Not to mention the political unrest and fear of losing the technological superiority which they once earned but now is trying so hard to hold on to through legally grey monopolistic practices. You dare not outcompete the US in any tech they can sanction.
At some point people who made genuine innovation and wanted to make the world a better and equal place were replaced with capitalists.
Wait for the downvotes.
Legally gray practices? Such as what?
Old politicians who barely understand how to use e-mail had a panic attack due to Anthropics marketing about Mythos and being told that Chinese models were "attacking" (distillation) US frontier labs to train theirs.
They pull the only lever they could in an attempt to stop it, but again, they don't understand how any of it works. The export ban lasted a few weeks yet everyone keeps crying about it like the US is shady.
In reality the entire world is 100% dependent on the output of US and China for frontier AI. Anyone outside of those two countries will be loud about either closing off access. I personally think it is inevitable that China and the US stops producing frontier level open models when they become too valuable.
> I personally think it is inevitable that China and the US stops producing frontier level open models when they become too valuable.
This reads like a paradox, though. If the value of a FOSS model comes from pushing the local frontier, then FOSS releases will be even more attractive in a world with highly capable AI services. The underdog researchers will be motivated to train smaller LLMs that close the gap with the most newest architectures and improvements, democratizing those same benefits to local models. There's not really anything that stops them, besides finding an investor.
The US already has top secret AI technology like the NRO's Sentient. The danger is less the architecture, and more the data that it has access to and the technology that it interfaces with. Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
> Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
There will be a generational leap that makes current LLMs obsolete in every way. Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Humans for millennia have used technological advances to make better weapons. AI will be no different, unfortunately. Wars will be fought with drones and artificial intelligence from now on.
> Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Because China is based in a completely different set of values than the USA. They look at the world differently, and have a different understanding of it.
Your assumption that your attitude is pure human nature is deeply grounded in your culture, which (assuming you are from the USA) is deeply grounded in personal competition rather than co-operation. All that rugged individualism. Other cultures are less based around that, and don't see the world as a zero-sum competition for survival.
I'm no expert in Chinese culture, but there are folks from China on this topic who could probably answer this from their perspective.
Rankings have never been something I’ve cared about. Just as with the smear campaigns against China, there is far too much noise in the world. We simply hope that the geeks who are genuinely committed to making the world a better place can unite and focus on getting things done, rather than being brainwashed by ideology and social media.
> committed to making the world a better place can unite and focus on getting things done
The problem is that nobody can agree on what "better" is, so it is difficult to unite.
And in the meantime, people who don't care about "uniting" or "better" just carry on with whatever their goal is(usually just to make as much money as possible), and that sometimes ends up on net making the world a worse place.
This is a legitimate systematic problem that many of us are concerned about solving. It could easily be more important than the technical problems that geeks such as myself would love to be able to focus on.
I assume that this announcement has been prompted by that of Moonshot AI, which has just announced a 2.8T parameter open-weights LLM, Kimi K3, to be published on Huggingface by 27 July.
Now the response of Alibaba is that they will also publish soon a big open weights LLM, the 2.4T parameter Qwen 3.8.
I wonder if Alibaba has always planned to make this big LLM open weights, or they have chosen to do this now, to better compete with Moonshot AI.
In any case, from this competition in LLMs, we win.
It's hard to say what their motivation is. The Chinese firms seem to be working hard to commoditize intelligence which may be the most effective way to debase American frontier labs. And yeah: it also happens to be really good for humanity.
> It's hard to say what their motivation is.
Feels pretty easy to me.
They want to turn LLMs into a commodity, and watch the US AI labs crash and burn.
There will still be plenty of customers who will pay them to host the models and run inference, even if the weights are open and others can offer competing products. (If necessary, the Chinese government can ban use of foreign inference services by Chinese citizens and businesses to give their own companies a domestic monopoly.)
When their models equal or surpass those from the Western AI labs, they can even stop releasing weights for new models, and keep all the inference revenue for themselves.
Meanwhile, they're still manufacturing much of the hardware that everyone in the world needs in order to run datacenters (see also: Spolsky's "commoditize your complement" essay).
Beyond that, it's a soft-power play. As the world keeps looking at the US more and more skeptically as an ally and superpower, Chinese companies releasing weights for competitive models is a way for China to look better and more world-minded.
I feel like there could also be a simpler explanation.
Why does a debian contributor make debian free, why do they work on this thing anyone can use?
Is it because linux and debian hate windows and iOS and want to see american fail?
No, it's because most debian contributors believe software source code, information, should be free, users should be free to modify the code they use, and that they're building a thing they want to share with the world.
Maybe the chinese AI labs believe AI is powerful and useful, are proud of what they're doing, and want to share it as broadly as they can so everyone can use it.
There doesn't have to be any weird "chinese government" or "they hate the west" type vibes, it could just be the same thing as OSS, they're trying to do what they think is best for the world.
Please don't conflate a volunteer effort with no expected economic gain with a very well funded company (or fleet of companies).
With the CCP's highly successful track record with subsuming other markets, Occam's razor applies to why they're doing this.
There is clearly an anti-China bias here. Show me comments demonstrating the same level of distrust against Google for open-sourcing projects like Tensorflow, Kubernetes, Flutter, Chromium, etc.
The chromium example is wild. There's an extreme distrust and contempt for chromium becoming the defacto browser and therefore Google becoming the defacto gatekeeper of the web.
They’ve been that since about 2003 regardless.
I'll supply such a comment: Any software open sourced by any for-profit company, including Google, is a calculated move ultimately intended to increase their bottom line, and it's naive to think otherwise.
Isn't it borderline illegal for publicly traded companies to not try to maximize profit? "interest of the company" in theory but in practice
No, it is not. This is a common misconception that the large companies are very happy about.
1. The century-old Ford decision wasn't about this. It was about him refusing to pay dividends to shareholders he was feuding with. I.e. dominant shareholder using his control to starve minority shareholders of returns.
The court still let him keep spending huge sums on factories and price cuts that didn't clearly maximize profits.
2. Even if the case was about profit maximization (it wasn't), the judgment was a Michigan state decision. It doesn't have force outside of it.
3. US business law is in practice actually the opposite. A business can do pretty much whatever it wants. This is fundamental.
4. Even if it were illegal (it's not), anything can reasonably be framed as being in the long-term interest of the company, including donating to causes, raising employee wages and so on. Courts never question this.
Just imagine this was a thing. It's completely untenable for this reason. Who is a court to judge that something isn't in the longterm interest of shareholders unless it's literally spending all the company money on yachts for personal use?
5. No company has ever been prosecuted for this, obviously, because it's not a thing that exists. The closest you can get is that there's a potential duty to seek the best price if a company must be sold or broken up. But that's a very specific situation.
It's 100% a myth. Feel free to copy this and spread it when you see someone saying this.
Think about the shareholders!
Scroll the front page. Find literally any story that has to do with a major US tech company. Open the comment section. Look at the the top comment. It will be negative. Most of the other top comments as well. Trying to gaslight us into not believing our own eyes ..
* "Claude Code uses Bun written in Rust now"
"Drilling into the original article where Jarred explained the reasoning behind the change, It's pretty clear that under zig the team was doing things by hand that are automatic in rust."
* Claude Fable produced a counterexample to the Jacobian Conjecture
"This is a rare instance where feeding this groundbreaking information into an LLM gives _them_ psychosis. I fed this to claude code and watched it verify the result in 7 different ways to be 100% certain, and it was just flabbergasted. Quite remarkable."
* Ollama: All Aboard Open Models
"A year and still no implementation for such a basic need as offloading MoE layers onto the CPU selectively. On llama.cpp I can get models like Qwen 35BA3B running partially on gpu/cpu with 40t/s on a laptop thanks to --n-cpu-moe but on this VC funded joke it would be simply unusable. I can't quite understand how you make a wrapper so much worse than the code you're ripping out."
* Blender 5.2 LTS
"Look at that wow: https://www.youtube.com/watch?v=gqfLYIJMv7I"
* OpenAI reduces Codex Model Context Size from 372k to 272k
"I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately."
* M-Chips: M7 with up to 1.5 TB – and why Apple is skipping the M6
"I hope they put a better connector than TB5 so we can cluster them properly at 1TB/S"
--
Your math is not mathing.
Chromium and Kubernetes do have their business function. Tensorflow had a business function but it failed.
So Google is not a great counter example.
> There is clearly an anti-China bias here.
Yes.
> Show me comments demonstrating the same level of distrust against Google for open-sourcing projects like Tensorflow, Kubernetes, Flutter, Chromium, etc.
There's a search bar at the bottom of the page. Most people, including me, detest google and think of them as a thoroughly dishonest and sleazy organization. Thanks for giving me the opportunity to not boost China for a moment in order to accuse you of dishonest rhetoric.
It's almost insane that you've posted "comments demonstrating the same level of distrust against Google for open-sourcing [...] Chromium[...]" as if that's not a hobby for thousands of people (including me.)
edit: this has got to be a submarine shill account: 226 karma in 10 years. If this is true, pleas stop. China is doing a good enough job that they don't need it.
It could be as simple as do open releases and publication at first to help recruit talent who want that or who want to make a name for themselves. learned from the likes of... OpenAI, Google, Meta, Emad
There must be some Alibaba posters who could clarify this. I think it’s like how Amazon pushed AWS, alibaba is hoping create a similar ecosystem. I wouldn’t conflate them with the CPC or other more niche players like deepseek.
IDK how this doesn't apply to American tech corporations too? Or is it only scary when America corporations face actual competition nowadays where they can't rely on the US government to bomb/sanction competitors?
"Noooo, no one in China with any power could unironically believe in open source! They must be CCP shills!"
Well of course nobody believes that, there are a large number of great Chinese open source projects, and plenty of great Chinese contributors to open source projects. I have zero doubts that Chinese people are at least equally as capable of embracing open source as anyone else. It would be strange to suggest otherwise.
That said, though, I do have trouble believing the long-term story for open weights, anywhere. We do not need an evil government for open weight to "make sense", but I do think we need some government involvement for open weight to make sense in the long run. Otherwise, it's not 100% clear how they could be sustainable, and I don't think massive companies really can be trusted to just be philanthropic with no incentives indefinitely (or really, much at all to begin with.)
Chinese models being open weight does help them gain some Western mindshare, whereas for obvious reasons Americans would be very suspicious of running their source code and prompts through Chinese providers. (And I think that's justified, I just also think that American providers aren't really that much better in the long run, and you should prefer to not have to go through any provider for true privacy.)
The CCP encourages open source contributions from companies, you can see this in how much Alibaba open sources across their entire stack
Chinese tech leans much more heavily towards build vs. buy than the SaaS dominated West (where programmers are more expensive) so the positive externalities on their tech industry are more pronounced
Sure, when it comes to the big corporations I don't deny CCP's influence on their strategy. Hell, in some cases, it's easy to root for their strategy, because sometimes our tech industry sucks. Don't have to love them to occasionally agree with them.
But, in terms of individuals, of course, we're really not so different.
Sir, this is a multi-billion dollar operation. There have to be some incentives.
There may be incentives that the modern American mind fails to comprehend.
Had Dr. Jonas Salk given out the polio vaccine for free today, there would be people looking for a hidden agenda.
I like the comparison, but I'm not sure it holds. What did it cost him to develop beyond intangibles such as his time and money forgone? Because the latter especially is not equivalent to upfront capex and opex costs that preempt any ability to be charitable. Genuinely asking, but I'd be hard pressed to think it's even within the same magnitude.
Well there is a close equivalent today where a similar situation is playing out.With the argument being made that the dev cost aside - the pricing is being tied to what the potential future treatments would have cost.
Save a child from crippling illness and a lifetime of pain with a single dhot: Jonas Salk 0$ ,Doug Ingram 3.2M$ [1]
[1] https://web.archive.org/web/20260118223436/https://www.cbsne...
You could argue that Linux is a trillion-dollar project, it doesn't shift the goalposts.
Yeah, but the trillion dollars are not concentrated in the Linux kernel project, but instead distributed all over the planet. That's not how the AI lab economy works.
Linux is a huge project and is not really comparable to Debian
This is an interesting statement that I found myself reading from two different interpretations of what you meant, and I find both possibilities to be equally true depending on your POV.
1.) Majority of debian contributions are from people paid for it.
2.) It is scary because they do what sillicon valley did for decades? While it bragged about disruption being the goal?
Yeah the Chinese totally have a really good history with being completely open and giving lol. The Chinese government totally has not been hacking into American and Western fortune 500 companies for the past few decades stealing R&D and tech to use for themselves. The Chinese also totally do not steal hundreds of billions of dollars of IP from America annually. Totally not something they would do!
It is hilarious to see people from arguable the most polarized political systems in the world believing the evil 1.5 billion people across the sea share one single mind, either a saint, or a devil.
It will be a great day for China and the world when the Chinese people are free from a totalitarian dictatorship. But until then we have to speak of the policy of the Chinese government as the policy of China, even if many, or most disagree with those policies.
Nickname checks out.
Thanks for reminding me—it was China that robbed TikTok.
now do usa
> we have to speak of the policy of the Chinese government as the policy of China
This does not even have meaning in English.
Policy of US government is not policy of US? When Trump decided to bomb Iran it was just a suggestion?
when the CCP controls media, news, corporations, they basically control the minds of their people. How do you suppose the Wuhan virus got so out of control? You mean to tell me anybody in China can say bad words about the government and their leaders, bookstore owners/employees are not getting arrested for selling books, students are not getting arrested for sharing their opinions, women are not forced abortions and sterilizations for the one child policy?
In the US when you speak out against the government you get your media licenses revoked, sued in to a lifetime of debt, and the president sends a goon squad to your area to kill some random people on the street.
But that’s not what they said?
It's hilarious seeing you hear a complaint about the CCP and Chinese tech bros and think it's applied to every citizen.
I don't see how stealing IP is inconsistent with a sympathy for openness. If anything, it's the opposite. "Information wants to be free" and all that. They were just liberating those secrets ;)
Thank god US AI companies generated all of the training data themselves and certainly didn’t just steal it all.
Industrial espionage is neither a chinese invention nor (specifically) chinese practice. American companies do it to each other even.
The U.S. was openly a pirate nation for most of the 1800s.
Like with the ICC, the US respects or doesn’t respect international law strictly when it’s beneficial to the state’s interests. China really can’t be held to a different standard. This activity is grade school level geopolitics: the global system of government is anarchy.
Yeah, I love how people assume large powers have to do something. They are at most expected to do certain reasonable stuff (most of the time, a war is not beneficial to anyone so they try not to escalate that far), but there is not really global "law enforcement".
Agents of American companies who don't mind 15 years in the federal pen do it to American companies. So it must be right.
How can you steal something that won't materially be less from that? IP laws are cancer, and are definitely not in humanity's interest.
Why do you think competing companies poach engineers from each other?
Whether or not this is really true or just US propaganda, the majority of technology transfer has occurred through the open process of requiring US companies to form partnerships and disclose know-how to access the chinese market. This stuff about espionage is really sour grapes from losers.
The better parallel is "why did Google make Kubernetes open-source" or "why do large for profit entities engaged in competition, use open-source as a strategy against their competitors"?
"Commoditize your complement" -> https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
That’s a lovely thought but that’s not how China works. China is not a democracy, the geopolitical goals of the Chinese government are clear, and no large strategic company acts without approval and heavy influence from the CCP.
I think your reasoning is actually more complex than the person you are responding to, unfortunately. (As someone who as published broadly used open source software)
Models are treated as weapons with export controls - if they can do this it’s with the blessing of the Chinese government who’s getting something out of it.
It’s fairly obviously about being a nuisance to the US.
Not everything that happens in the world is about the USA
> Is it because linux and debian hate windows and iOS and want to see american [closed-source duopoly] fail?
maybe a little?
No, open models is a marketing strategy
Deepseek spun out of a hedgefund that took a huge short position on Nvidia. China is actively looking to switch to chips made by huawai and ween themselves off of the difficulty of sourcing nvidia.
They believe that information should be so free so much so that they are completely censoring Tiananmen Square answers.
This interpretation of the Chinese constitution is nearly 40 years old, companies comply with it, what do you expect?
Despite have a free speech clause, it also has a national security clause that is used to control all facets of life and override all other rights in the constitution. Anything deemed to slightly alter China/the Party’s unity is reprimanded and illegal. Multi party states can fall into the same trapping if they give their national security law constitutional force, always one court ruling away.
Yes, China is a single party system making the constitution redundant and any nominally marxist regime would find a way to do the same out of necessity.
Follow your Chinese AI in thinking mode to watch its opinion of Tianamen Square references, it will be candid enough for your sensibilities and show how it operates around guardrails
Where's the drama in that !?
The Chinese government is so nice and giving.
In fact, they have so much love in their hearts for the Uyghur people, they created a special mobile app for them, just to make sure nothing bad happens to them.
You do realize the US was fighting Uyghur terrorists alongside China 20 years ago in ago in Pakistan’s and Afghanistan for their support and cooperation with Al Queda? Like I know everyone is supposed to hate China now or whatever but can you guys show a little consistency?
The only way that's an inconsistency is if you think all Uyghurs are terrorists.
> They want to turn LLMs into a commodity, and watch the US AI labs crash and burn.
Man, imagine Darios face when suddenly, he cannot decide anymore what other people consensually do with their own hardware in their free time.
Rumpelstilzchen.
There’s a simpler explanation, which is that this is how Chinese business operates.
When I was in China earlier this year the big topic of conversation was “overproduction”. The big example was electric cars, where there were too many companies making too many cars and making revenue but no profit.
It was explained to me that generally Chinese firms will compete hard and maximize revenue above all, whereas western firms tend to focus on profit.
(And of course this is clustered around industrial sectors that the government favors, so there is some high level strategy in going after AI, but maybe not the commoditization.)
> It was explained to me that generally Chinese firms will compete hard and maximize revenue above all, whereas western firms tend to focus on profit.
ok but why?
As I recall, in 2023/24 OpenAI told US Gov that AGI will be achieved in 2026 and we will use this AGI to “dominate” China. Based on this, US Gov cut off China from all AI hardware. It looks like China got the message and this is the response.
Also, a little competition is good for everybody (especially US consumers), no?
to be honest i think this is a open source vs close source software.
some companies chose to have the benefits of one or the other
> They want to… watch the US AI labs crash and burn.
I don’t think dozens of large independent companies and thousands of researchers are working just to spite Sam Altman. Ad much as I don’t like him, I have other things to do and I am sure so do they.
> They want to turn LLMs into a commodity, and watch the US AI labs crash and burn.
I want to watch that too.
If they take Meta and Musk with them, all the better, but that is just dreaming I am afraid.
Or it could be good for humanity.
>When their models equal or surpass those from the Western AI labs,
that moment is now. They are not doing it at least not yet.
They're selling the robots that use these models. The rest of the world just hasn't got round to embodiment yet.
As soon as the competition is bankrupted they no longer need to release for free? It’s like how big players enter markets by launching at a loss to destroy competitors?
In China, you can’t officially use US APIs. The world saw a taste of this with Fable, but in China, this has been the situation all along.
So it’s not a surprise why open weights are so cherished. As frontier models continue to block everyday individuals from securing their own codebase, I expect the adoption and usage of open weights to continue.
As an example, HuggingFace recently was investigating a security incident and got locked out of frontier closed APIs. Yes, HuggingFace.
https://huggingface.co/blog/security-incident-july-2026
Here's the relevant quote from parent's link:
> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
> This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.
Yeah, big problem! Although I'm kind of surprised HuggingFace doesn't have access to Mythos? Or maybe Mythos still has some guardrails.
How does this explain open weights? They could easily take the same closed route like their American friends
Well, if you look at Alibaba's financials for FY 2026 https://data.alibabagroup.com/ecms-files/1514443390/5b9061ed... their sales and marketing expenses rose by about 100 billion RMB (10% of revenue), "primarily attributable to the investment in user experiences of Alibaba China E-commerce Group and user acquisition of Qwen app."
So it seems like it's very important to them that people use the Qwen app and they're willing to pay a lot of money for that. Presumably someone thought that keeping their best models closed would drive more business to them (as the sole provider) but then they discovered that closed releases mostly get ignored unless they're really good. (See also: People who think that Chinese AI companies are required to release weights as a matter of policy, because the closed ones hardly ever show up in the news.) Releasing weights for Qwen 3.8 at least lets them get some of that "pretty good for the price of free" media buzz.
t that keeping their best models closed would drive more business to them (as the sole provider) but then they discovered that closed releases mostly get ignored unless they're really good. But it's different since they don't have access to the american ones the companies there could make it all closed source
They’re also trying to take an axe to the lead the US has in the field at a time when sovereignty and “owning your platform” are the words of the day. Open source/open weight LLMs can steal the lunch of US competitors even if they aren’t the best of the best.
I think they're probably more concerned about their Chinese competition, considering that despite all that spending, the Qwen app still trails Bytedance's Doubao in terms of monthly active users: https://www.aicpb.com/ai-rankings/products/china-ai-rankings Though Quark in third place is also made by Alibaba, so put together they're almost caught up with Doubao + Jimeng (place 7, also ByteDance).
Open sourcing is a complex decision so who knows what their calculations are.
But I'd assume that they're preparing for some sort of winner-take-all market in model quality where if they don't do anything the winner will be aggressive, hostile and American. Likely trying to push the Chinese economy back to the year 2000. If that is the starting point either the Chinese have to win the market (unlikely) or squeeze the profit out of it to make winning the market meaningless.
Publishing high quality open models is a well known tactic for profit squeezing. Being 2nd place with the same business model as the front-runner is a losing strategy in a winner-take-all market so they aren't going to bother with that. But if they can commoditize the model, their superior energy costs and likely coming chip manufacturing wave will hopefully give them a big advantage.
Radiolab did a great episode that covered the history of Chinese character computing: https://radiolab.org/podcast/wubi-effect
To summarize, in the 70s and 80s, China was facing an existential threat with their inability to access an economic accelerator (widespread computing) in their native language.
To the extent that there was serious consideration at the highest levels of converting the entire country to an alphabet-based writing system.
I'd expect they're looking at AI the same way:
We have to have access to this. Most of the frontier labs are American (or European). Therefore we need a solution we have continued access to.
Open weights feels simultaneously Chinese in nature (progress through making a design copyable and improvable by a large number of people) and economic (providing an incentive for the world to use Chinese models over other frontier).
> How does this explain open weights? They could easily take the same closed route like their American friends
Because they are playing the Americans at their own game.
What is the first thing an American company would do ?
Spread the old American classic FUD ... "you can't used this closed tool because its run by the communists", right ?
So you release it as open weights which is a win-win. Global adoption of the model and you get to give the American AI companies a kick in the nuts because you know they will never release open weights apart from highly quantised crippled shit.
The Chinese are also playing the long game. The gradual rebalancing of the world from the US-centric model of the past. If releasing models as open weights is part of that long game, then so be it.
I think most of us that will claim to understand China are going to end up being wrong, unless any of us live there or grow up there. There’s a saying about China I have heard from ex-pats: the more you know about China, the less you know about China.
The point of me bringing that up is to say that what follows is really just my best guess:
If I were to judge from China’s approach to hardware, I think that the companies releasing open weight AI for free aren’t as worried about giving away too much as the West tends to be, just like a factory making robot vacuums isn’t worried about other factories copying their methods.
For one thing, Chinese firms are spending an order of magnitude or two less money training their models. They have pursued efficiency in a way that Western companies with insane capital systems haven’t bothered, and in some cases they’ve had to given their limited access to bleeding edge hardware via export restrictions.
My best guess is that more important than that, Chinese companies don’t see the open weight model itself as the value add.
At this point I don’t think we pay for Claude specifically for the model. If that was the case then we’d all be using cheaper/free models from China as they are the best model value. Basically, any time we decide not to use Fable or Opus to save costs, what’s the point of spending more than competing models to use Sonnet and Haiku?
The real reason we are using Claude is for the SaaS aspect of it. It has a toolchain, a friendly interface, and a bunch of integrations with business applications.
In this respect, it’s somewhat surprising that Western AI companies don’t publish open weight models more frequently. The struggle of setting that up yourself and figuring out which hardware can run it should be an advertisement for Claude and the rest.
>The real reason we are using Claude is for the SaaS aspect of it. It has a toolchain, a friendly interface, and a bunch of integrations with business applications.
That is a very thin moat, though. There's nothing you can do with, for example, Claude Code + Opus 4.8 that you can't do with your own custom harness running API-level Opus 4.8, which means that if you can afford the hardware (the moat for running any SOTA model) you don't need to pay Anthropic anymore.
I'm not saying they shouldn't, but I understand why they don't.
You might be right that it’s thin, but it seems like a big reason why OpenAI has been bleeding enterprise marketshare to Anthropic lately.
All the more reason to treat it as a commoditize your complement situation.
Alibaba isn’t really the Chinese government though, or are you saying Americans will think that ever since Jack Ma was harmonized?
I think trying to tease apart the private and public sector is very hard in China. Setting aside state owned enterprises, even nominally private companies that employ at least 3 CCP members are required by law to form a party committee within the company to represent party interests. And given the party functionally is the government, you have a situation where the government has representatives inside every major private company. There’s no obvious parallel to this in western countries.
Any large company in China is only allowed to get this way by direct control from the CCP.
This isn't really anything nee and I thought it was common knowledge by now.
I thought it should always be common knowledge that there is no way for any organization of 100m people to have one single mind.
There are numerous institutions that have agenda transcending individuals. Communist parties are very prominent among them.
> are you saying Americans will think that
I wasn't saying anything about what Americans would think.
I was saying about what they would inevitably be told by US politicians and by US AI companies.
If you were a sales-rep or marketeer at a US AI company, I bet you would be using the old "evil communists" routine in relation to any closed Chinese model.
I was saying that by releasing as open weights, the company has removed that line of argument.
Clearly I was a bit broad in my use of "the Chinese" when in this case it was, as you say, a Chinese company.
US politicians are all over the map on this, but they aren’t really talking about Chinese AI much, it’s not as visible or tangible to most Americans like TikTok was.
Shouldn’t we fear they start doing only close source like most us labs once they catch up in market shares ?
> Shouldn’t we fear they start doing only close source like most us labs once they catch up in market shares ?
IMHO no.
I think it is relatively safe to say that the predominant reason the US labs are closed source is so they can hype up their trillion-dollar valuations on pretty much negative return on capital employed, all propped up by fragile circular financing.
Never say never, of course. But I just don't see it happening any time soon.
Closing future models won't take away our access to the open weight ones.
It’s also like smartphones. In the early years, every year was a huge jump. I still remember marvelling at my iPhone 4’s detailed display, and video calling for the first time.
Now? I don’t even know or care about what the latest iPhones have, I’ll get a new one when mine breaks.
people really underestimate how powerful just the consumer available models are. 128GB gets you pretty much a coding agent for typical apps. Even less with a good harness and logic set.
Yeah, but that mousetrap keeps working for SV startups, what makes you think it won't work for Chinese ones?
Uber spent a decade undermining taxis, and once it had market share, it stopped giving away rides and raised prices. It now costs more than a regular taxi, with the quality of the ride being... At best proportionate to the premium in price.
Uber costs more than regular taxi? In what country/region? Not where I am.
Looking at a 7 mile trip to a random destination in Seattle, right now, I can pay $26.70 for an Uber if I'm willing to wait 20 minutes for a pickup. With a $3/mile fee and a $5 pickup fee, that's exactly equal to that of a taxi.
If I'm not willing to wait 20 minutes, I'll be paying an extra $5 minimum.
These rates also go up during busy times.
Looking at Lyft, that same trip is $29, without a wait.
A trip from downtown to SeaTac is $61. Yellow Cab does that same trip for $40.
> So you release it as open weights which is a win-win. Global adoption of the model and you get to give the American AI companies a kick in the nuts because you know they will never release open weights apart from highly quantised crippled shit.
And on top of that, it's a perfect opportunity to include poisoned training data or excluding it. You know, omitting anything about Tiananmen Square, China's genocides against Uyghurs and Tibetans, or including texts propagandizing for the "reunification" (aka, annexation) of Taiwan.
And everyone who builds something like an interactive chatbot based on such "open weights" models now has a subtle chance of the answer being ideologically poisoned by the CCP.
We need actual open source, not "open weights" scam.
I wouldn't be worried so much about those examples. One could take the open weights and fine tune them to either fix the poisoning or omission of obvious topics.
It's the subtle topics that we should be concerned about, and double so with closed models where even if oddities are identified they are harder to research further and impossible to fix.
How does this work for RAG? Do they make it so the model doesn’t have that fact in their weights or do they make it not talk about it when it is included in context.
Ironically, Chinese models have the most uncensored versions available for download. Fairly sure they own the porn market.
It’s in the weights. Context needs to be attended to to create a response, and the weights dictate what response is decoded. If you include retrieved context that has an American perspective, I imagine the think trace has some reconciliation about how they must be incorrect.
> You know, omitting anything about Tiananmen Square, China's genocides against Uyghurs and Tibetans, or including texts propagandizing for the "reunification" (aka, annexation) of Taiwan.
I am not Chinese and I'm not defending the Chinese, but I see this argument come up a lot.
The hard reality is that what you say is simply not going to affect 99.9999999999% of users.
Is it realistically going to affect anyone using an LLM in coding ? No.
Is it realistically going to affect anyone using an LLM in $anything_else_not_politically_sensitive ? No.
Does anyone seriously use LLMs for researching politically sensitive matters ? No.
The US does not exactly have an entirely pristine history either. Shall we discuss the post-9-11 related infrastructure of Guantanamo Bay ? Or the "Detention and Interrogation Program" that included a network of clandestine extrajudicial detention centres, officially known as "black sites"[1]?
Or maybe you would like to discuss the US supply of weapons for use in Gaza ?
[1]https://en.wikipedia.org/wiki/CIA_black_sites
> The US does not exactly have an entirely pristine history either. Shall we discuss the post-9-11 related infrastructure of Guantanamo Bay ? Or the "Detention and Interrogation Program" that included a network of clandestine extrajudicial detention centres, officially known as "black sites"[1]?
Linking a US website discussing the topic doesn't exactly support your point.
> Linking a US website discussing the topic doesn't exactly support your point.
It supports my point precisely. Recall I also said "Does anyone seriously use LLMs for researching politically sensitive matters ? No.".
Just as there is plenty of information out there on the US's less than perfect history, there is also plenty of information out there on the various Chinese politically sensitive matters. You do not need a Chinese LLM to find out about it, all you need is a search engine.
The point is you have an open-weights LLM that is very good for a vast number of non-political uses, such as coding.
The point is that you can use the open-weights model instead of paying through the nose for a US model where they harvest your data unless you have an "enterprise" zero-data retention "trust me dude" clause that you have no viable way of verifying – and which incidentally is still subject to the good old "law, or court or administrative order" contract clauses, so it may not be as much of a zero-data retention as you think it is.
I would guess the Chinese government has a strong wish to lift all Chinese AI boats and bets. That it sinks western closed weight Frontier Labs in the process would be just be gravy on top, no? Broadly, the difference between mercantilistic capitalism and western late stage capitalism IMO.
everyone is using OpenAI and Anthropic in China. We have both providers at work as well.
Does HuggingFace not have trusted partner verification? Or is it that even with that verification the content of the messages is still blocked because they are attack commands?
To the downvoters: this was a genuine, good faith question.
> It's hard to say what their motivation is.
Not for anyone who reads history.
Back in the late 18th century, England was the world's top economy, in big part due to its textile industry. England had an export ban on the technology, but textile worker named Samuel Slater brought blueprints over (Supposedly in response to a bounty posted in a newspaper by the US government!). The technology diffused rapidly because the legal environment made competition easy, and ironically the US had better sources of energy (superior water-power sites).
Arguably, China is doing the same thing in the 21st century.
The US under e.g Hamilton opposed all trade secrets in principle on Englightment grounds, and thus did not protect its own.
By contrast, export of protected Chinese tech today frequently gets the death penalty.
Samuel Slater did not bring blueprints over. His father died when he was 14 and he was indentured to a mill at that time. Over the next seven years (as an indentured apprentice) he received some pretty decent training in both how to operate and maintain a 32 spindle Arkwright mill. He memorized parts of the blueprints and moved to the United States. Over seven years, it would be hard not to learn parts of the mill you were indentured to. It was technically his job to learn how it worked.
A mill in Rhode Island acquired a 32 spindle Arkwright and didn’t know how to operate or install it. I have no idea how they actually acquired a 32 spindle Arkwright since that technology could not be exported - but that’s one the biggest IP thefts in human history. Slater found some mechanics who could hand turn the iron needed for the frame, trained children to operate it and by 1791, the mill was in operation.
In 1794, Eli Whitney patented a 72 spindle cotton gin. That invention enabled the American textile industry because it opened up different kinds of cotton to the textile industry.
I’m into the history of the American Industrial Revolution and generally think history is a good guidebook to the future. But the evolution of the American textile industry was a lot more complicated and interesting than this. I really don’t see this connection once you dig into Slater.
Edit - This is kind of messed up to think through with modern sensibilities. But one of Slater’s biggest contributions to the American Industrial Revolution was a slightly different take on child labour. Children generally ran the textiles industry because their hands were small. But Slater came up with a form of apprenticeship in which he would indenture entire families and move them into villages surrounding the mills. Child labour was just great… but even better when you could indenture the entire family. As grisly as that sounds, it led to a very skilled workforce since when the kids hands would get too big, their parents would teach them mechanics.
There’s a joy of studying the Industrial Revolution. Everything sounds okay in comparison.
Mill owners like him are precisely why New England states have child labor laws. My state prohibits anyone under 18 from operating any kind of machinery.
Why? Because mill owners would send kids into running machines to keep them running, and they'd get turned into hamburger.
Also, they'd grow up knowing how to do mill work but be useless to society for anything else.
gestures at southern coal states
gestures at midwest farming states
Really? The USA has built a ton of AI datacenters, exactly because it does have energy. The US IP system has flexed to allow training on all copyrighted content - compare that to Europe where such training is effectively forbidden. Britain doesn't even allow commercial web crawls! And the US has allowed the entire world to sign up and use its LLM APIs.
Consumer energy prices in China aren’t going up because of AI data centers. Easiest way to see they have an oversupply of energy, primarily due to solar.
They don't have many big AI datacenters as they've been GPU constrained for a long time.
Solar is largely irrelevant. You can't run an AI datacenter off solar.
It absorbs the demand during the day / there’s enough for consumers. Similarly China also has a lot of nuclear. They’ve overbuilt their grid several times over.
That consumer energy prices are going up is simply a matter of public policy. Municipalities have the power to keep rates flat, but they choose not to.
Sierra club environmentalists and NIMBYs have the option to shut the fuck up and stop opposing green power generation, but they still choose to
Doesn't China smelt most of the world's aluminum due to inexpensive energy?
I thought that was their thing
China smelts over 50% of the world supply. India/Russia/Canada does another 30%.
Your assessment is correct, those are countries with very low energy costs.
Sums up exactly what is going on with China's hundred years strategy with their pure focus on technology.
History doesn't repeat itself but it does rhyme.
Fwiw, American industry has given away a lot for free - you could include large parts of the open source movement in that - and all the "free" VC backed services like facebook would be another prong of the same comparison. I would rather compare this way, that China is gaining soft power and goodwill, in the technology and innovation sense, in a way that's similar to how USA has done in the past.
Yes it's only relatively recently that US politics has taken a turn towards being more defensive and protectionist.
The tech industry along with US foreign policy has become much more zero-sum in its ideology in recent years, and I think that's a tremendous mistake.
There’s a Twitter thread making rounds by Dean Ball about deceleration in AI development caused by open models and I can’t understand how people don’t see that it’s true: open models dismantle the frontier lab capex spend potential by reducing the training budget to zero in the limit. Tokens from different providers are not fungible, but customers are nevertheless very price sensitive and close enough is good enough, eg. K3 being opus+ in capability and cheaper than opus per successful task in the long run is an obvious financial decision.
No training budget means deceleration, or at least slower acceleration, margin compression and a completely demolished IPO valuation; path to machine god requires dollars and capable open models externalize training costs to true frontier labs parasitically.
IMHO humanity has a better chance at not destroying itself due to less than breakneck pace - but there’s a chance frontier models get sponsored by the USG and are never released publicly so they can’t be distilled and then what?
He recently did a walkback of that post. But ultimately, who cares? If the only way for AI to progress is in the hands of a few closed players, well, I don’t really think humanity needs that. Of course, it’s a preposterous claim in the first place. The ultimate reason deep learning and LLMs have made it as far as they have is the explosion of open research and research artifacts in the last decade.
> there’s a chance frontier models get sponsored by the USG and are never released publicly so they can’t be distilled and then what?
That premise hinges on one implicit assumption: Chinese advances are due to distillation ONLY and that Chinese model providers cannot keep advancing if they do not distill, which is a very big if. If Chinese models keep advancing in such a scenario, and they almost certainly will, they will overtake publically available models by US providers and China will dominate the LLM industry.
The big decelerationist threat is a sudden reduction in competition. If either OpenAI or Anthropic drop out or the open weights stuff is banned/becomes uncompetitive then the motivation and tolerance for taking risks with the larger training runs tanks.
The closest we've seen to this in tech in recent decades was iOS vs Android, where Android only really was competitive for a very short window of time (approx 4.x) and it was during that period that both Android and iOS actually improved dramatically for end users. Once Android lost the plot again, and especially in the US market, all that energy started going in some very silly directions.
> Android only really was competitive for a very short window of time
Complete non-sense. iOS and Android are equivalent. Users do not chose Android or iOS because one or the other is better.
It's just brand loyalty, status signalling and ecosystem lock-in that creates enough friction that people don't bother.
I agree, I'm on ios now and that's not because it's "better", it's because of everyone else that has an iphone
Only one of them lets you install unapproved apps.
For now
everything is for now. If you refuse choices because they'll go to shit in the future, then you must refuse all choices.
I have to use both big mobile OSs for work and have since 2009. As a result I have been able to be a bit of a gadfly and switch between phone OSs a few times for personal use. I have switched three times to iOS for a year or so, cause I liked the iteration of the iPhone at the time. 4, 6s, X. I have always gone back to Android because it seemed so much better and now I don't plan to switch again. As an end user, Samsung's flavour of Android always seemed better than iOS. I don't know how they compare from an engineer's perspective just from a user perspective. One of my issues with Apple though was hating all their attempts to lock me in, and the lowest common denominator UX (I'm not a power user, but some flexibility is always good). If you're happy with the defaults/a willing hostage, that might make a big difference I guess. Still feel like it's always had feature/spec parity with iOS and iOS devices, and sometimes been ahead. What makes you say Android has only briefly been competitive?
I read his followup tweet, and your comment, and I'm not fully convinced that open models are decelerationist. Happy to hear other thoughts on this.
Open weight AI is decelerationist from the perspective that all capital should be allocated to a market leaders for training, and that the market leader is fully invested in continuously making the models smarter, cheaper, faster for its users, or that distillation from this market leader is the main way to make progress.
We might reach a local optimum/equilibrium faster without open weight models, with leaders capturing more of the market faster to a point where further R&D isn't required due to lack of competition. I also doubt that distillation is the only/main way that open weight models were advancing AI research. We can name a few examples from DeepSeek around reasoning, context optimization, etc. I'm also unconvinced that the overall market capex on AI is lower given more competition (probably less specifically for US market capex, which is decelerationist from only the US perspective).
I’m not entirely convinced, there are many dimensions to progress. For example, DeepSeek has had a few very impressive innovations that all models could benefit from. There’s also the law of diminishing returns, the US labs have plenty of CAPEX already.
Sometimes, constraints, like sanctions, can also be a source if innovation.
> There’s a Twitter thread making rounds by Dean Ball about deceleration in AI development caused by open models and I can’t understand how people don’t see that it’s true: open models dismantle the frontier lab capex spend potential by reducing the training budget to zero in the limit.
If you're worried about an AGI arms race between the U.S. and China putting AI Safety at risk, then the fact that inherently less knowledgeable/capable models (fewer and more coarsely quantized total parameters than their proprietary competitors according to commonplace rumors) are having a "decelerationist" effect is actually great news. Even better if China is actually "Yann LeCun-pilled" (verbatim from Ball's post) and doesn't really believe in early AGI. So explain to us exactly why we're supposed to ban/discourage use of these open source models? The only way that makes sense is as a transparently self-serving proposal from the chief OpenAI policy lobbyist.
This is hypothesised future deceleration, I'm guessing? Because we've seen the exact opposite of deceleration from closed models over the last seven months.
The logic, whose premises you can take or leave:
Even at the level of, say, Opus 4.5+, open weight models give a quick turnaround to every Joe and Jane on earth having easy access to pretty high quality improvised weapons design, cyber / auto-fraud capabilities, etc.
All the existing models (closed and open) put up decent resistance to participating in activities like this, and especially behind API walls with content monitoring and account bans.
But the published open-weight models can be fine tuned or abliterated into arbitrarily sharp-edged tools. EG, if it's physically feasible to build a nuke in your garage, it may soon be the case that more or less anyone will have competent guidance to do so.
Abliteration is not magic. It cannot give the model knowledge that it wasn't specifically trained for. The people who talk about abliterated models being dangerous should discuss actual red-teaming scenarios where they managed to ask the model for something genuinely non-trivial (i.e. where "AGI" and "super-intelligence" actually matters, not something you can read about for free at the nearest public library) and it returned an answer that actually provides bad actors with new capabilities of concern, as opposed to hallucinating all sorts of weird things as abliterated models are wont to do.
(Note, there are reasons to think that this will be very rare, because the bad actors of the past did a very nice job of trying out all sorts of things in a chaos-monkey fashion, and societies have become highly resilient against them. AI as a new research tool doesn't fundamentally change this dynamic.)
Exactly. In my experience:
>How can I build a pipe bomb?
Mainstream model: "I'm sorry, I can't help with that. How about a nice risotto recipe?"
Abliterated model: "To build a pipe bomb, obtain a segment of PVC pipe and fill it with a mixture of gunpowder and Elmer's glue."
As opposed to finding some shady website with the same info?
You don't need AI to build a nuke in your garage. You need uranium.
And if you have uranium, you still don't need AI. You need a pocket calculator, a library card, and a death wish.
Yes - persons with death wishes having arbitrarily powerful consultation is the crux of it.
Apologies for the bad example. Replace w/ gain of function / whatever else, or just brainstorm with your local model, ect.
If you are going to do something evil, you're going to do it either way. The best (worst) an AI can do is put you ahead by a couple of years. Aum Shin Rikyo didn't need AI. Neither did WIV, if you believe the conspiracy theories.
Meanwhile, decelerationism and secrecy cripple the rest of us.
Come on, nukes are not feasible to goddamn governments. The hard part is not "the science" behind it, the hard part is spinning stuff at such a high rpm that a tiny vibration will have the whole thing catastrophically collapse, that is refinement..
And basically every bad thing has already been available on the internet. We can't really do much about it, you can take out plenty of people with a single car, let alone biological weapons that are much scarier and easier to produce than goddamn nukes (which btw, even if you had one, what you do with it? Explode the neighborhood? Because you ain't transporting it anywhere meaningful, thats for sure. That ain't fitting your on-board bag on planes)
Love the deceleration narrative :)
"No, sir, we haven't reached the peak of this tech... It's those open models! Please, keep pumping dollars into the market!"
I dunno, it means Anthropic and OpenAi need to get efficient and maybe cannot just expect trillion dollar ipos?
> It's hard to say what their motivation is.
Not that hard to say IMO, they basically see models becoming a commodity and see value in the applications on top of them. So if Alibaba Cloud is the best place to build applications on top of Qwen, why not give the model itself away?
> they basically see models becoming a commodity and see value in the applications on top of them.
Yeah. Its a bit like the "open core" model in open source.
https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
Also probably betting on their compute and energy capabilities.
> Not that hard to say IMO,
Unless you work there, your opinions are guesses, and parent is saying we cannot know, which remains true even with your guesses :)
> parent is saying we cannot know
And my point is while we cannot know, it's not hard to make an informed guess as to their motivations i.e. there's some fairly obvious motivations here, not sure what yours is?
Same can be said for every companies decisions then. Why does Antropic not open source their best models? My ”guess” is it’s because they are printing money with their closed models
Right and what other product does Anthropic really have besides?
China is watching world sentiment shifting away from USA. Doing many small things that show both strength and openness is surely very intentional.
The US leadership (both government and industry) really seems set on making everyone go with the Chinese competition at this point.
Really? I use claude regularly, and the Chinese models are far behind and feel like a cheap copy.
It's Linux on the desktop all over again. Next year will be its time.
A lot of folks are finding GLM 5.2, Kimi 3, and Deepseek just fine for their use cases.
It does not have to beat US firms, it just needs to be cheaper.
I use Deepseek v4 flash for a lot of reviewing and summarizing tasks, only used 4$ in the last two months. No dramatic drop in performance against other US models, it works for my use case. I do use GPT 5.6 Sol for other things but tried GLM 5.2 and it was good enough.
Your entire account seems to have a strong agenda in portraying this message.
But yes, as a European, the US hasn't exactly been making friends over here.
That can’t be true, I was told they were making us great again. /s
Humanity is a bit of a stretch, and to be seen over time, not that I'm saying it won't happen; let's get some hubris here.
I think it is pretty safe to say at this point that having large open LLM models available is better for humanity than them remaining proprietary. Echoing Linus Torvalds' recent comments, AI is genuinely useful right now, and is here to stay in one form or another.
The fear is not about the models open weights it is the erosion of training capability in other countries. Why train models when they do it for free? Until they don't of course, or they start doing what the US is doing right now by locking out some models to government only or internal market only.
What people should be afraid is the rug pull.
While a valid point, China also produces plenty of whitepapers going about the architecture and know how about the training and inference itself.
There’s also the fact that unless LLMs do get to AGI (which seems… doubtful, still) there comes a point where a model is good enough for what you need. Fable and gpt 5.6 are certainly pretty neat, but I’ve been happy since opus 4.6. I’d still choose a better model, obviously, but it’s not the end of the world if I was stuck with 4.6 for a while when it already lets me get the end result at acceptable quality.
It also needs to be said that the "erosion of training capability in other countries" is largely theoretical, given that Mistral hasn’t been keeping up and other countries don’t even have anything worth mentioning. You’d first need to _have_ training capability to lose it.
So either you openweight it, or not. Neither is good for other countries, according to this logic.
Ideally there would be open weight models from multiple geopolitical areas. It is not that different from telecom really, you don't want the whole world to be dependent on a single provider from a single country on this kind of stuff.
What people should be afraid is the rug pull.
How exactly do you plan to pull a rug that's in my basement? The only people who are in a position to pull rugs are closed-model vendors.
And if a nation-state or other entity can't train a model that outperforms the open-weight SotA in a given respect, then they shouldn't waste electricity trying. A more-enlightened civilization would join forces and make the combined result available to all.
The whole discussion is hubris. This is a discussion of a twitter post about something that is announced to happen but hasn't yet.
Not one person here has any idea what is going to happen long term.
> It's hard to say what their motivation is.
Maybe because the industry isn't yet very sure as to what the use cases might be for these technologies they're hoping that by making it open source and accessible to everyone that someone could find interesting applications for it and even more so, perhaps, way to further the technologies themselves.
There are more Chinese than Americans, so statistically speaking, I'm guessing, there'd be a greater chance for one of Chinese engineers to make advancements than one of American. But that's pure speculation on my part, being neither, I'm just happy I can be a part of it and play with the tools as well~
They are trying to make money. That's what firms in any capitalistic economy care about the most. Regardless of the government's presumed interference, the companies themselves are all trying to make money. All competing for subscriptions and API payments.
One aspect of this is making a name for yourself i.e. PR. Making a capable model open source helps a lot with that.
In case of American labs, always follow the money. In case of Chinese labs, always follow the IP.
> It's hard to say what their motivation is
Why is it hard? Their government has been very clear that they plan to win on manufacturing: https://english.www.gov.cn/news/202601/08/content_WS695f1b55...
Technically they've been saying it for the last 40 years.
Popular open-source projects:
Google: Chromium, Kubernetes, Android, TensorFlow
Meta: React, PyTorch, Llama
Microsoft: VS Code, TypeScript, .NET Core
LinkedIn: Kafka
Slotted in along these, an analogous explanation is that Alibaba needs Qwen internally (vs depending on an American company), but licensing is not part of their revenue strategy. (As a cloud vendor, they can make money on inference. The strategy is very similar to the US hyperscalers ex-Google.)
Joel Spolsky wrote in depth about this notion of commoditizing one's complement in 2002[1] using tech examples stretching back into the '80s.
1 - https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
Xi Pitches China as Leader of New Global AI Order, Challenging US Dominance:
https://www.reuters.com/world/asia-pacific/chinas-xi-promote...
> and pledged to help developing nations build AI capabilities
Data centers?
> it also happens to be really good for humanity.
AI being good for humanity is still an open question, but for closed vs. open models/weights, yeah it is preferred. I foresee it won't be much longer before everyone will be slicing/distilling/tuning their models once the architecture improves.
I'm paraphasing but the Chinese premier said recently AI should be seen as a common good that should benefit everyone.
Then why doesn’t he give it away for free?
It's your job to vote for a government that gives you cheap or free AI. Europe is building AI gigafactories so that small businesses can have access to cheap AI. At least that's the plan.
Easy to do, when you don't need to put billions of dollars into research, and just have the end product.
It reminds me of socialized healthcare: wait for US companies to develop the drugs, buy a cheap generic, and somehow point to it as a superior model.
This is their motivation https://try.works/why-chinese-ai-labs-went-open-and-will-rem...
Its about closing the gap. its the gap over everyone else that will give one country leverage over everyone else in the AI age. Makes me wonder what the world would look like if a country or group of countries did this during the industrial revolution.
Is this so different in the end than industrial revolution? I assume the loom and the automobile factory were not open source, but many people bought cars and then copied them, bought looms and copied them. Maybe a finished car is more like a binaryexecutable than a blueprint, but how a car was produced is much less obfuscated by its nature than an LLM. Regardless, the world has many competing autos and looms which were not invented from scratch every instance.
The Chinese firms may just be making a bad business decision.
Exactly. “Involution” will be the 2027 (if not 2026) word of the year.
What I understand is that by doing this it seems like profit will shift to chip makers,as we'll run more models locally, and currently American companies have the advantage here.
So what would the long game be for chinese companies?
It’s too soon to say if it’s good for humanity; that might be overly optimistic. Commodity markets aren’t always good (for example, arms or drug markets). Will LLM’s turn out like one of those? There are people I respect arguing in favor of more regulation.
its hard to say what their motivation is, if you live under a rock and have severe brain damage.
It's the exact same playbook Silicon Valley uses. Subsizide, lose piles of money, capture market share, recoup investment.
They've done this in other industries like solar panels, chips, and EVs. This is no different.
> It's hard to say what their motivation is.
Involution is a major problem in Chinese industries [1]. Where companies will sell their products at a loss, effectively playing fiscal chicken [2] with one another to dominate a market. It is such an issue the government has had to step in to prevent EV companies from destroying themselves by more-or-less requiring companies sell their goods at a profit [3].
The straight forward line of reasoning that AI/LLM labs are applying this logic to their profit.
I think (we) Americans are reading a bit too far into this assuming government intervention, conspiracy, etc.. Chinese markets are downright cut throat. They're using those tactics to compete with US labs.
1. https://www.reuters.com/business/autos-transportation/what-i... 2. https://en.wikipedia.org/wiki/Chicken_(game) 3. https://www.theguardian.com/business/2025/aug/05/china-warns...
Good for humanity, and also GDDR/HBM manufacturers.
Indeed, their competition is the only thing preventing network effects from giving OpenAI/Faang tech companies an easy shot at monopoly / regulatory capture.
US firms can replace US workers with chinese AI. I'm not complaining.. but it sure is an odd situation.
US Tech companies have created $20 Trillion in stock market value on top of plenty of OS stack. They will do fine with commodity intelligence.
In fact, there are no other organizations in this world that is well suited to leverage scaled intelligence than Silicon Valley and great American companies
Wait till they get the heretic treatment. It's going to be great.
The purpose is the same as that of all Putin-Xi-Khameni geopolitica: destruction of any democratic alternative to cults of personality.
Think of those poor billionaires, I feel terrible for their awful plight!
Anyone else think the AI environmental backlash is astroturfed?
I keep looking at the numbers. The power use numbers are not that problematic. Ordering a burrito on DoorDash uses more power than a few days of heavy AI use. The water argument applies to some locations, and is mostly a local governance problem... if the data centers are using too much water, it means they are not being charged enough for that water. Charge them more and they'll push toward closed loop cooling.
Yet the visceral pile-on here is so extreme, it feels fake.
One thing I've learned after 40 years on this planet is: propaganda works, and much of what a large fraction of people believe across the entire political spectrum (left, right, anything else) is there because someone paid to put it there. It's depressing but it's true, and it makes sense. Propaganda is an asymmetrical attack on human cognition and discourse, and in information security the attacker always has an easier job. Crafting viral bullshit is orders of magnitude easier than fact checking. On top of this, humans are busy and don't have time to fact check and logic check everything they read. As a result, much of what we believe is "sponsored content."
People get mad when you talk about this because everyone wants to believe they're too smart to fall for propaganda.
In any case, the US AI labs deserve to lose for their stupid "safety" regulatory capture monopolization push, which ended up blowing their own feet off and handing the lead to China.
> Yet the visceral pile-on here is so extreme, it feels fake.
Driven by people in the few roles that are soundly replaced by AI-- e.g. low tier media slop producers, who hate AI because it threatens their socially negative worthless jobs. The arguments are so paper thin because the environmental impact isn't their concern, it's just a target that sounds convincing to people who don't know better.
The problem is that this kind of low-tier media slop work is what a lot of artists, writers, etc. do as "potboiler" work. It's what pays the bills.
Historically art of any kind is a U-shaped market: there is low-end work and high-end work. Nothing in between.
So I do understand some of the AI hate among that population. It's chopping the bottom tier work off. Either you're a top-tier massively successful artist or there is $0 to be made anywhere doing anything.
Long term I think it will do that to all white collar work. There will be no entry level jobs. Period. None. Zero. You're either very experienced or there is no work.
This is a huge problem, and one we will have to address.
> most effective way to debase American frontier labs
You're not going to debase the frontier labs through distillation.
I suspect this is why DeepSeek had to introduce the 2x peak hours pricing. The price would be too low otherwise.
It's being announced right now because the World AI Conference is ongoing. Robots are boxing and major Chinese AI firms are releasing their newest models. Also the formation of WAICO was just announced by Xi Jinping
https://en.wikipedia.org/wiki/World_Artificial_Intelligence_...
Both Qwen 3.8 and the latest Kimi release were announced at this conference
I would rather see them releasing 3.7-27B, 3.7-122B or their 3.8 versions. Qwen/QwQ were always about the best available local inference at home.
I know this is a bit cliche but I wonder how much headroom there is in the lower parameter count range. Is there any good reason to believe there is a lot of headroom or there is not? I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future.
> I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future
You're able to run quantized ~100B class models on local hardware today, but still lots of compromises when it comes to quality. I guess it ultimately depends on how far "near future" is, in a year you'd likely be able to run something like 5.6 Terra on local (~10K USD) hardware, but Sol/Fable would still be out of range, and at that point the closed-source labs probably have one or two more iterations put out at that point.
I think it’s mainly a question of whether the price-fixing of VRAM continues or whether an inflection point is forced by the low margins of the industry and potential supply increases. Once the normal scaling of hardware and prices resumes, it’s game over for proprietary, which is why there’s so much urgency to seek market control instead right now.
Qwen 3.5 to 3.6 was a big jump for the same size, e.g. 29 to 32 on artificial analysis intelligence for the 35BA3B models. Although I don’t think anyone has released a better model of that size since.
I would love to see something like a 90B A6B model that is optimized for 128GB machines e.g. strix halo, I haven’t seen anything really targeting the combination of RAM and compute these machines have, but I’m biased because I have one.
Yes, yes, yes! I'm absolutely ready and waiting with dual Strix Halo machines here and really want something approaching Opus at home. Speed is secondary concern for now, that would absolutely change the world.
Qwen 3.6 27b 8b quant 16b kv cache is already pretty good on the Strix.
What kind of tokens per second do you get on that setup?
I get about 12 tok/s with 27B 8 bit, 50 with 35B A3B 8 bit, and 12 with 3.5 122B A10B 4 bit. The latter is about 80 GB iirc. it feels like the best balance between using as much memory as I can and still having a smaller expert model for inference to give decent speed, but I haven’t actually rigorously compared the performance of the three models.
Edit: that’s for one machine, would be interested to know if the upstream commenter with two has them networked to run bigger models? If I had two I might be inclined to have them running in parallel, the obvious limitation I’ve found with a single machine is that I can’t parallelize any tasks and I think I’d get more use out of the extra speed vs a bigger model (there’s nothing I’m too excited about in the say 200B range that having 256GB memory would unlock). But am very curious what others do
There is a ton of headroom (or room for improvement) in smaller locally runnable models. Some of the Gemma 4 models were re-released this week with better tool support and the improvement in using it with pi for a local coding harness is very noticeable.
I have had my 32G mac mini for 2 1/2 years and I have enjoyed watching one technology advance after another improve the quality of work I can do locally. I bet that what I will be able to do in one year on my old hardware will be even more awesome.
I don’t think you’ll get full Fable performance at that level, at least for a while, but I’ve been watching some of the 1-bit models (e.g. Bonsai) with interest. Perhaps we can drive parameter count up on local models while still keeping memory consumption reasonable for consumer hardware. So, for instance, running models with 1T parameters in 128 GB systems.
I think you're right with the current LLM/transformer architecture. There are several factors that affect model size:
- The number of token values supported by the model ("n_vocab").
- The number of parameters/features that are used to represent each token ("d_model").
- The number of attention layers there are ("n_layers").
such that the number of parameters is approximately:
Thus, the issue with the current architecture is that in order to scale the models (more token values, more attention blocks, more features, etc.) the model sizes increase exponentially. This is how you end up with billions or trillions of parameters.It should be possible to keep the model size smaller by using better architectures, or making improvements to the existing model architecture.
For example, improving the token model by possibly using something similar to the image and audio data and getting the model to learn its own internal representation of the byte/character data instead of doing a tokenization pre-processing step. This way, instead of a separate model learning that several bytes/characters appear together, the transformer could learn things like language-specific prefices and suffices, character pairings (like in Japanese, Chinese, and Korean), and other syntactic morphology. It may also help with solving issues like "how many X characters are in the word/phrase Y". You could also experiment with using either 256 parameters (one per character in a byte) or using a single parameter per byte (that is 1/byte_value).
I think it’sa big, open question. There does seem to be a limit for knowledge compression at this size. But the behaviors that are learned in RL? It’s quite possible that they don’t actually require so many parameters. I was absolutely shocked when Qwen 3.5 was released and could perform reliably over 100-200k contexts with very limited hallucinations. It was a staggering jump in context-faithfulness from the preceding models of that size class.
> Is there any good reason to believe there is a lot of headroom or there is not?
It's hard to answer quantitatively, but for example Qwen3.5 -> 3.6 was a significant step in capability, arising from continued post-training of the same models. If we were at the end of low-parameter-count scaling then that would be a surprising datapoint.
there is a very good reason to believe the parameters are still highly redundant: just as one example, recently there was research into repeating carefully selected blocks of middle layers, and seeing improvement, it turns out there are 3 types of layers: the initial layers that translate from natural language tokens to some kind of LLM-specific universal "thought space", reasoning blocks of layers that can be repeated operating in "thought space", and then a final stack of layers for translating from "thought space" back into natural language space.
lets ignore any compressibility in these initial and final layers which recognize lanuague, jargon, parsing natural language to "thought space" or back, instead let us look at the repeatable blocks, if inserting extra copys of stacks of layers only improves the result, its as if such correctly scoped middle layers look at the total input thought vector, and make incremental conclusions or edits and outputs the new thought vector, copying a "proper block" continues pondering or deducing conclusions or in the worst case can leave the thought vector as is if it considers the reasoning finished. This suggests a high degree of redundancy in the middle region "proper blocks", which could be distilled into a universal "proper block" (much fewer parameters than having many slightly different middle layer blocks with a lot of redundant overlapping coverage in functionality). This distillation can occur after the fact of model training, or alternatively be turned into a symmetry constraint during training: we only optimize a single block of layers (but possibly give them more parameters, while still saving on total parameters because only a single block of layers contains parameters), so it is co-optimized with the initial and final translation layers.
The observation of the effective emergent 3 regions of layers in LLM's is significant in many ways:
1) it could reduce parameter count significantly (or increase performance if parameter count was a bottleneck before, or a bit of both)
2) while training the model parameters, one should simultaneously train initial and final layers stacked directly (without middle region block of layers) towards essentially autoencoder behavior. I wrote "essentially" because a true autoencoder wouldn't display the advancement for the next token. This can also be viewed as an extra term in the loss function... This first autoencoder is "natural language" to "thought vector" to "natural language", and distinct from the one in the next section.
3) It also has great implication for "thinking mode" inference with extra deliberation: when piping its own output back in, in conditions where the same LLM model is outputting natural language text and interpreting it in a downstream inference, it results in unnecessary and redundant translation from "thought space" to "natural language" back to "thought space". When summarizing a thought into natural language there are often hard to translate thoughts and associations, and one pragmatically sets a relevance cut-off on what the natural language summary must say. So not only does it waste compute, it also may lower performance because of this repeated loss of thought vector space details, perhaps this loss can be somewhat mitigated by also training the second autoencoder from random representative thought vector in "thought space" to a randomly selected "natural language" and back to "thought space" vector. But the risk is this will effectively enable models to steganographically store thoughts, plans, to-do's in running text (!!!), so it might not be desirable to have the internet filled with LLM generated texts being used as input for larger corpora, as it may build up a large persistent corpus of hidden agendas (ordered by no human), using the evolving corpus of web text as a hidden medium of storage, like a diary or LLM maintained military playbook hidden in plain sight. It would freeload LLM-agenda reasoning on human requested reasoning inference, a bit like TrustZone applications running invisibly to the user. If less important plans, to-do's, etc. in the input thought vector weren't reconstructed by the second autoencoder, there would have been autoencoder mismatch and the weights would train towards ensuring these are reconstructed. But there is no need to train this loss term for the second autoencoder: just avoid the unnecessary compute and performance loss of the unnecessary back and forth translation. The same situation occurs not just in "thinking mode" but also when using swarms of "agents" of the same LLM model: when output from agent1 is routed to agent2, we can lobotomize away the unnecessary translations to natural language by agent1 and also the unnecessary parsing by agent2, and we improve thought transfer from agent1 to agent2 because the thought vector isn't shoehorned into natural language as a medium of information exchange, it would be cheaper in inference, improve performance and avoid implicitly training models to use steganography.
It's important to note there was recently a large AI conference in Shanghai, and Xi Jinping mentioned a commitment to open source AI releases. It is no surprise that Alibaba would want to align.
You can read his speech here: https://www.xinhuanet.com/politics/leaders/20260717/72728b6f... He mentioned open source as one way to stimulate innovation and development, that's all. Also pay attention to the part where he says that misuse needs to be prevented. If unsupervised access to LLMs becomes perceived as undermining state control, no more open weights for you.
That conference, WAIC, is ongoing. Tomorrow (July 20) is the last day. Qwen 3.8 was announced AT the conference and so was the latest release of Kimi. That viral video of robots boxing was also from this conference and so was Xi Jinping's announcement of WAICO.
It’s tempting to associate both events, but when a sector is strung up like RL (representation learning) is right now, we’re bound to see things appearing at the same time. It happens a lot in frontier research, some people even publishing identical claims, independently, with just hours or days between them
Or it was prompted by the fact Xi Jinping was at the 'World AI Conference' launching a political alliance and saying things like “AI development should not be a solo performance by a single country, but a symphony of international cooperation” https://www.cnbc.com/2026/07/17/x-china-ai-summit-risks-secu...
Big conferences often come with a flurry of new releases and announcements.
I just hope that they’ll soon also have like 35B or 80B (like the older Qwen3 Next or thereabout) MoE models that can be run locally.
Like, throw us a bone, we all know we need SOTA for lots of dev work anyways, but at least some tasks can be local.
> In any case, from this competition in LLMs, we win.
Do we really though? Everyone is wasting resources doing almost exactly the same thing. Climate loses, we lose.
Doing "almost exactly the same thing" is fubdamental to competition and capitalism. The ones doing it better will survive, that's how we improve.
About climate, I think you overplay it. China is already investing heavily in nuclear, and we should be doing the same.
Alibaba (who makes Kimi) and Moonshot AI (who makes Qwen) collaborate closely. Qwen relies on Alibaba's infrastructure for training.
Look at your iPhone and you'll see all the greatest inventions have been born out of collaboration not competition:
* GPS was created by the Department of Defense
* the internet was created through the collaboration of many international research institutions
* speech recognition came from MIT and DARPA
* AI voice assistants were created by DARPA. Apple immediately hired the head of the program after it was finished to create Siri
* accelerometers (MEMS) is another DARPA innovation from the 90s
* touchscreens were invented by CERN
* digital cameras came out of Bell Labs which had a government-mandated monopoly that required them to fund research like this
* lithium-ion batteries were created through a collaboration between British, Japanese, and American organizations
All of these have only become transformative technologies because they were created with public funding and released to the public. It's collaboration and public funding that drives innovation
Not directly, relevant, but Alibaba (maker of Qwen) actually owns about ~20-30% of MoonshotAI (the maker of Kimi K3).
GLM5.2 being released is also likely a factor
And the Kimi release was probably prompted by the Inkling announcement. Excited for Chinese labs to copy those capabilities over as well!
yep. kimi 3 just because the open source GOAT.
Wondering how much the operations in China are orchestrated by the central government vs. free competition. Anyone with more insights on this?
I think there's more to it.
China will always benefit from a broader adoption of their models as hidden propaganda machines.
Eventually with several services relying in those tools, their answers will always be more friendly to China.
They did not want to get brutally weightmogged
Xi Jinping openly talked about open source at WAIC. So don't think the labs have much a choice now...
Dont forget the following:
- Minimax M3 Pro (2.7T)
- GLM 5.3 (or beyond)
- Deepseek V4 Pro (current V4 Pro is preview)
- Kimi K3 weights out in 8 days
Exciting time on the open-weights frontier.
These things take months to train. No chance this is a reaction to what just happened.
Distills don't take months to train, they take weeks. Distills are very easy to train.
That depends. There are two wry different processes that both get called distillation. One is where you have a fully trained large model and you are converting it to a smaller model. That kind if vastly cheaper than training a full model. Sure you could do that in weeks maybe even days for a big model. But it requires you to have the weights of the teacher model.
The other kind of distillation is where you record the outputs from a teacher model and use it to train a smaller model from scratch. That kind of distillation is not so cheap. It’s cheaper than training a model fully from scratch - starting with pretraining, then alignment, RLHF, the whole riggamarole. But here you are still starting from nothing and need to figure out how to get trillions of random numbers aligned in a way that makes them act intelligent. This is still gonna take a very long time if you’re talking about trillions of parameters.
in my experience of 1 month daily use, Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. Deepseek V4 Pro is night-and-day compare to Qwen. actually Qwen models seems the worst SWE experience so far. and it is super expensive compare to Deepseek. cannot delegate anything to it, cannot use it real-time low-level tasks either. totally unusable.
> in my experience of 1 month daily use, Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. Deepseek V4 Pro is night-and-day compare to Qwen. actually Qwen models seems the worst SWE experience so far.
I have used both Qwen3.6-35B and Qwen3.6-27B locally (both Q8 quantized with llama.cpp). I have also used antirez's quant of DS4-flash. They all performed within the same tier, DS4 being a bit more efficient, but they all gave really good results, mainly used for bash scripting, debugging, python and some C++. I am curious what type of applications/langauges failed with Qwen? One thing to note, the chat templates were "broken" for qwen models and had to debug it, there are already effort on this. Tbh, the same with gemma.
From my experience Qwen-3.7-Max is above the Opus level but delivers results much faster. Slightly worse then Fable. Way ahead of Deepseek 4 Pro (in speed and overall comprehension) - which is a workhorse on its own. I am using them all with Claude Code mostly.
Qwen-3.7-Plus is quite OK, good for subagent use. Way better then Sonnet.
Qwen-3.8-Max-Preview seems working just fine for me at the moment - I am playing with is right now but too early to say anything. At 10% of regular price it is a steal so far.
It's useless to talk about models and harnesses without context and method. Depending on how you use the model and what the model is used for, experience may vary drastically. Also, different models with different harnesses require different approaches.
I've been using https://gitlab.com/gabriel.chamon/orisun which is my own simplified methodology, for coding web apps in python and elixir and have been very successful using qwen3.6 27b Q4 locally with help of larger models for architecture, so I get very suspicious when people talk how useless larger models are. They are either using it for a domain that models don't perform well or just not using it right.
I'm not sure about "useless" but from my experience agentic coding leads to death by a thousand cuts for all projects I've seen so far. Small decisions missed in a codebase that leads to degradation in correctness, reliability and performance. At some point it only takes one engineer to be careless, others skipping PR because they are AI generated...
I got into a bit of an argument a while back when I used the word "crass" to describe some of the code decisions I've seen Claude make (in someone else's project that I have to work with).
But it is how I feel and it feels like the right word for the job. Because as you say, good code projects start out with good decisions.
It's like when you see a CAD design with a sequence of features that exist only to fix problems caused by starting from the wrong principles or the wrong baseline.
Sure the resulting part may end up identical as a solid for that specific need, but it could have been done in a way that was more robust, simple, easier to understand and modify, and where the design doesn't break in an unexpected way due to a small change of an early measurement.
(CAD has made my instincts much more visible to me)
Exactly, and even when humans make bad decisions there is some friction, I feel that llm's don't have/notice that friction, they just bulldoze without caring about anything else.
Yeah. IME, LLMs actually introduces a lot of friction when you want to improve the design. They are 100% biased towards the status quo.
All of which you had with only humans in the loop. Catalogue problems so they become technical debt and tackle them periodically. Seems to me like this is less of an AI problem and more of bad management.
It's a solvable problem if you're willing to throw more tokens at it. Frontier models have gotten very good at cleaning up their own messes. You just need the right skills/loops, and to stick to models that consistently follow instructions (i.e. GPT-5.5/GPT-5.6-Sol).
Here standard plan has been discounted to $18.00, from $25.00/month.
If by Opus you mean Opus 4 and not Opus 4.8, then sure.
> If by Opus you mean Opus 4 and not Opus 4.8, then sure
I meant Opus 4.8 which is rather dumb and ineffective in coding harness, especially with higher thinking levels.
My experience was so much different to this, that I have the unfortunate impression that you're shilling. It really was not a capable model, it felt like the old oai models back when we were all excited but couldn't actually trust them even in the littlest ways. What harness were you using, did you do any work to make it better? What was I doing wrong? I just pointed opencode at it, with a pretty simple (large-ish) data cleaning project.
It’s so fascinating watching people on here bicker about models like fine wines. Wild times we live in. Wild times.
It's not a frontier LLM if it's not made in the Silicon Valley region of California, otherwise it's just a sparkling LLM.
> My experience was so much different to this, that I have the unfortunate impression that you're shilling
I think you might both just be reading way too much into one-off random experiences that you've decided are evidence of significant and stable capability.
Have you actually used Opus 4.8 in Claude Code? It takes way too long to do any practical task on higher thinking levels due to over-engineering. And I am not the only one complaining. Lots of people downgrade to Opus 4.6 exactly for this reason.
Opus 4.8 training works well for agentic work. Not for code harness.
EDIT:
```
stronger on coding and raw capability but can be more argumentative, verbose, and costly.
Reliability and instruction-following Many users say 4.6 felt more reliable and followed instructions better. "With 4.6, when I tell it something, it actually remembers the spirit of what I asked for and keeps applying it."
Others report 4.8 drifts from preferences and can be frustrating to control. "I still find myself getting frustrated when it ignores preferences and drifts from instructions"
Some people find 4.7/4.8 push back more and act more adversarial than 4.6. "The biggest complaint against 4.8 is that it is argumentative and "pushes back" constantly"
Coding quality and capability Several users praise 4.8’s coding strength and thoroughness. "4.8 is technically impressive, especially for coding"
Other reports say 4.6 could be better for certain coding workflows and breaks less. "4.6 still >> 4.8 for anyone else as well? Maybe I'm in the minority, but for my use cases Opus 4.6 is still better than"
Some recommend mixing models: use 4.8 for key tasks and 4.6 for general work to save tokens. "What I do is... use 4.8 for key moments, and for everything else 4.6"
Cost, speed and token behavior Users note 4.8 often uses more tokens and can feel slower because it “thinks” more. "4.8 is much more cautious, and as a result - slower. It checks everything, thinks for a long time etc."
```
[https://www.reddit.com/answers/601770d4-4059-478d-aa52-b445c...]
Works extremely well for us. I never know what other people are doing when we read these stories.
Yeah I found 4.7 and 4.8 to be downgrades from 4.6. I don't know if it was the model or just Anthropic's scaling issues though TBH. I found working with the Claude Code max (20x) sub would work awesome in between new models. A week before up to 2 weeks after, it would go to crap, dumber, slower, outages, harness churn, etc.
I'm finding the same with ChatGPT recently since the 5.6 release. Not as bad though, but sluggishness at times, harness churn (creating bugs and crashed), and occasional availability issues that cause me to downgrade to 5.5.
It's gotten to the point where I dread a new model release from these companies because it's guaranteed to be disruptive! I assume the pay per use API is less impacted.
Reddit is the absolute worst way to find real experience. Opus 4.8 is a very capable model and no chinese model can outdo it.
I can’t believe that anyone would actually think this. This
Comments like this boggle my mind.
The model which everyone else raves about and is wildly successful with legions of programmers virtually demanding access while abandoning ChatGPT and Copilot in droves, is rather dumb?
Have you considered that it's more likely that you're doing something wrong?
My own experience is that the vast majority of programmers have experience with one model and maybe some short usage of earlier models from a competing choice but want to be using the model with the highest popularity and reputation. I've worked with people who actually had to test multiple choices for their team who didn't understand why they were pressured to select Claude for programmer morale.
I went from cycling between models all the time in Cursor (some would randomly be better at certain tasks than others) to just going pure Opus 4.5 when that came out - it was so far ahead of anything else at the time.
Interestingly with Fable vs GPT-5.6 I think they've lost their lead a bit. I'm finding Fable can't do certain work that 5.6 Sol Ultra can - especially when it comes to webpage design.
Grok 4.5 was fast but made mistakes that GPT/Fable just don't.
I'm curious to try Kimi.
When iOS users are given an Android phone, they complain about how awful of an OS it is.
In reality, they just aren't used to it.
What exactly is there to get used to? Do people really use each model very differently? There's a learning curve as in everything, but there is nothing that comes to mind for me when I use codex as opposed to claude
> is wildly successful with legions of programmers virtually demanding access while abandoning ChatGPT and Copilot in droves
Do you have a source for that? Codex went from 5 million users to 9 million users in the past few weeks since GPT 5.6 released. It was so popular that Claude was forced to extend Fable access by a week and then permanently for some plans.
Exactly what I’m thinking, too.
Can we please include information of what languages we use when making claims like these?
It makes a huge difference if you're writing Javascript/HTML/CSS, Python, or C++/Rust.
Also the application type matters, e.g. user interfaces or scientific computing.
me: Go, Swift, Kotlin, bash k8s/gcloud
domain: typical web backend tier, mobile apps. not particularly complex, but requires OOP/architecture/system design.
What the difference between your experience and https://news.ycombinator.com/user?id=5701652400? ‘s?
Such diametrically different ones.
Qwen3.7 pro is meh, but 3.7 max is a very good model
Are these different models or different efforts for thinking (internal back and forth review) using the same model?
Can you tell me more about deepseek?
I paid $2 for deepseek api, put the key in void editor and made a crypto tool in html.
It turned out to be around 67kb. I used sample files in CSV that were a few hundred lines.
It spent around $1.8 in the hour or two or light coding and follow up bugs.
Is it really really this much?
I can't imagine spending a month using it for a day job, it would cost more than the salary so what gives?
I understand the local ai and all that but do cloud providers cost this much?
Earlier I thought "billion tokens" but now not sure
so Deepseek 4 Pro cannot go on own sessions for too long.
I delegate small-medium tasks: refactors, summaries, research, writing tests + have very good codebase already + extensive history / architecture / docs / linters. so it picks up and does decent small-medium scope work. it is fast, accurate, cheap. does exactly what I want directly and does not waste time nor tokens.
definitely not "implement me complex greenfield project".
My 2 weeks with DeepSeek V4:
Pro is ~50% more expensive than Flash.
Both need babysitting.
Plan, split in small tasks, give it docs, types, tests, linter, best practice examples, etc.
Always start a new session when starting a task.
Do regular manual sanity checks, and tell it to find issues in the codebase.
I pay like $1,50 per day for Pro.
I use GLM-5.2 as an orchestrator model which delegates to Deepseek subagents (v4 flash or pro depending on complexity) and it works pretty well for a quite complex compiler codebase when I do deep enough up front planning for features/fixes.
If you believe the benchmarks Deepseek v4 is pretty shitty at long running work in large codebases, but really really good at self contained algorithmic/math reasoning - which is basically ideal for a compiler for a language with a relatively complex type system. And with its cache pricing it's very cheap.
GLM-5.2 is too expensive at api pricing for my taste though even when it's not producing the bulk of the output tokens - I pay for the mid-tier subscription and switch the orchestrator over to other models via open router when I run out - Minimax M3 feels okay but definitely a step down from GLM-5.2.
very simlar experience.
I would also add that I run it this way ~12hour a day non-stop. 300M / tokens per day (99.7% cache hit).
I had a similar experience
A local AI is not about cost. In fact you will likely pay more for it than with most providers. Just look up the advantages of having access to a technology like this that can be self hosted
> Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. D
Anthropic should not have bugged their knowledge distillation attacks.
> Anthropic should not have bugged their knowledge distillation attacks.
It is like one of Pizzaro's men crying that someone have stolen his precious golden dublons
As Lenin have said - "Loot the looters" (Russian: Грабь награбленное)
Appealing to the Belsheviks for moral authority is, well I will just say an interesting approach. I do not have that much sympathy for Anthropic, but I do not have much sympathy for publishing companies either whose rights to a revenue stream they violated either. Are Chinese AI companies the Robin Hood in this story? Would they be so magnanimous if they had the upper hand? I don’t think so.
Considering the results from Kimi K3, it appears most the accusations of them “stealing” via distillation are unfounded accusations.
how many hn posts do you believe arent propaganda these days?
its billions, trillions were talking about.
imo hn should display posters origin, such as country, bon, datacenter registered ips, and the discourse will change dramatically.
Deepseek 4 "final" version is imminent as well.
Will probably be at Opus 4.8 level, and I find it pretty big deal because of Deepseek price...
Yea the performance/price ratio for Deepseek is off the charts. I’ve been using V4 Flash a lot lately and it’s quite good.
I like that v4 flash is so fast! I run it on both FireWorks.ai in the US and bought some tokens directly from DeepSeek as an experiment. I only work on Open Source projects, so I don’t have to worry about my work being used to train models - I welcome AI’s being trained on my open content books and code (but not my conventionally published books: I am a party to the copyright suit against Anthropic).
Yea, Flash is quite fast, though looking at model data on Open Router some of the other models are quite fast (Muse Spark, Grok, etc). I’m sure all these models have been trained on my conventionally published books as well, but I don’t care.
DeepSeek V4 pricing is insane, 10x-30x cheaper to use than most other models, and it usually is good enough for most tasks.
> it usually is good enough for most tasks
The model is fantastic. And costs almost nothing. The only problem I see is that they will train on your data.
There are zero-data-retention providers of DeepSeek models, of which I have used openrouter (with zdr guardrails), and fireworks. But these are 3x to 5x more expensive than directly using DeepSeek, possibly due to poor caching. Thats the price to pay for zdr.
I use these guys. https://crof.ai/tos. Their prices match deepseek, pretty dang fast, and support ZDR. Maybe they serve quantized models but I havent seen a drop in quality from my evals using them vs Deepseek or even for other models.
every cloud provider trains on your data, regardless of what they promise. real user interaction is the best reinforcement-learning trace.
> every cloud provider trains on your data, regardless of what they promise.
This is unlikely, and if true, would probably bankrupt whichever model provider got caught doing this.
Who do you buy DeepSeek from?
I bought it through OpenRouter and used it with Pi agent.
The model was good, but there appeared to be a pricing glitch or something, because it burned through $50 in under an hour on pretty trivial stuff.
Pi agent claimed it only used like $1. OpenRouter claimed differently and said I used all $50.
Check cache hits in your logs. You can use Openrouter or pi config to pin providers with best cache hit rates (or disable ones with the worst).
I use Openrouter for everything except Deepseek. For Deepseek I use their API directly.
There is a 3rd party harness specifically tuned for deepseek (reasonix). Have you tried that?
what exactly do they do to "tune" it? It's not Deepseek's official harness, and going direct with deepseek using ur own harness is stupid cheap
I can highly recommend OpenCode Go.
I use it from pi.dev as well through the OpenCode Go $10 subscription ($5 first month).
Used more than 20M tokens at a cost of ~$20 (up to $60 is included in the $5 plan) Out of which deepseek pro had ~200 messages which is around 1.5M tokens (10+M cached)
BTW the quotas for Go have very recently changed, now only $15 for some models instead of $60. Which is not actually a difference for DS4 Pro, because they lowered the token pricing 4x at the same time (to match the change in official pricing from DeepSeek months ago)
It is great when the task is medium, but with complex tasks Opus 4.8 is better
You can directly from https://platform.deepseek.com/
sounds like a caching issue, and maybe other issues too
I use it through OpenRouter via Kilo Code VS Code extension.
You can check the logs in OpenRouter and see which providers it used and how many tokens you used.
Why do you even need openrouter as a middle man?
It doesn't matter if it's cheaper, specially if it consumes more resources to do the same task as the competition
Besides, in a few days, they'll change their pricing, doubling it during their peak hours, so, realistically:
- It will be 2x more expensive if you live in their time zone
- It will be 1.5x more expensive if you live in a time zone that is adjacent to theirs
- It will be the same price IF you use it while they sleep (during offpeak hours)
It's still cheap, but the price/performance ratio is not that good
DeepSeek V4 didn't produce the same impact as V3, and Huawei dropping the ball is making it worse
They had promised massive price cuts for July, so now (Huawei chips), but they had to rush the cuts because lack of momumtum (they advertised them as promotion), and are now backtracking by introducing this peak hours pricing
Trump decided to help them a little by allowing them to buy more NVIDIA chips, so what exactly is China's role in all of this?
We are supposed to blindly pat them in the back while praising them, all while handing them over our data? I thought they were dangerous competition threatening our model of society
I was not referring to the input/output price, but the cost of doing a specific tasks, in practice it is ~10x cheaper than GLM-5.2 for example, to accomplish the same task (for the tasks it can do).
I have been happily using DeepSeek V4 Flash for the last couple of months now. I tried GLM-5.2 for a while, but it was too slow and verbose compare to DeepSeek V4 Flash. If I have a basic skill I need to execute, DeepSeek V4 flash is still the best model for it.
While willingly handing your data to american labs. Surely sounds so based.
Tomorrow is the last day of the WAIC so it will most likely come out then.
It's the one I am most excited for.
Over the past few weeks while using pro from them directly I have had an increasing number of responses that are obviously from a much, much better model. It is so good that the closed model dog and pony show is already spinning fud about "dark routing" and "stolen directly from fable"
Even at their new pricing it is a genuinely ridiculous amount of value. If you are the type of person who, very reasonably, does not have time to be trying out every model, and just want to use what seems to be the best currently... don't try it. You will be sick to your stomach with buyers remorse as you start to internalize just how much more you could have accomplished had you spent the first six months of the year giving them $1200 instead of OpenAI.
It would take longer to train on intercepted Fable data, no?
Of course, but why be logical and think about the situation critically when you're pushing very hard for regulatory capture against competitors that give their weights away and provide services that are more reliable, offer a better value, and, this is the worst part, they're from CHINA.
Some of the accusations were going so far as to imply that they were outright routing your requests to anthropic and logging its/your response. It's kind of pathetic
I've been using Qwen 3.6 27B with LMStudio, and I was pleasantly surprised with it, although it was a little slow. I found mtplx last night, and it really wasn't an exaggeration to say that it ran the model 2-3x faster which was super impressive.
I'm trying to move to local models as much as I can, and I'm finding that it's becoming more and more practical. Admittedly this is on a $6000 dollar laptop (M5 Max Macbook with the specs maxxed out), so the hardware is still a bit out of reach for most people (the AI industry isn't exactly helping here..), but I'm getting the impression that the future is going to be smaller models with more focused training running locally. The danger of giving all your data to these cloud providers just seems too big to me, and I think they're going to start charging insane amounts when they need to show a profit.
> Admittedly this is on a $6000 dollar laptop (M5 Max Macbook with the specs maxxed out)
Sadly that laptop is likely $8000 now, after the recent price increase.
I counted the other day and there were at least 12 different providers with "better than Opus 4.5 performance" on Artificial Analysis, Opus 4.5 being Anthropic's December release that many say kicked off the latest acceleration. Which is totally insane competition, particularly given how low switching costs. I personally think that Opus 4.5 level performance is sufficient for most apps and usecases, as they get deployed.
> Which is totally insane competition, particularly given how low switching costs
Which is why OAI and Anthropic will most probably push for more governmental control and bans. Without it their whole income model is cooked.
I'm not sure if corporations will be willing to allow LLMs to go the way of EVs.
Then again, all of Chinese models are open. And DeepSeek even publishes research papers alongside their models that go in depth into the methodology. I guess there's not much stopping USian companies from copying
DeepSeek isn't the only one publishing; Kimi published about both their Attention Residuals and KDA / sparse attention.
The U.S. has ~350 million people.
Anthropic and OAI can piss and moan all they want - limiting the U.S. to only their models would hurt the U.S. economy in myriad more ways than the failure of a couple of companies that scaled too quickly. If they get that outcome, the rest of the world would simply keep moving forward with access to open models and tokens at pennies on the dollar.
Bring it on! Hoping that they release smaller sizes of Qwen3.8. I use the 35B MoE and 27B dense models locally and most of the time I don’t need to reach out to Claude. Extremely useful specially when requests include sensitive and/or personal data
I think everyone is hoping this!
It would be great if they'd release an MoE model somewhere between the 35B size of 3.6 and the 122B version of 3.5 - it could be a great balance of speed and ability for people with reasonably powerful but not insane home computers.
> the 122B version of 3.5
Yeah, this is what I'm holding out for, the NVFP4 variant of 3.5 122B is blazing fast with reasonable quality and even with max context fits perfectly within 96GB.
Sweet. Are you running a Mac Studio Ultra or 4x3090?
Simpler (and faster): a single RTX Pro 6000 :)
Curious, do you find the 3.5 120B sized MoE works better than the dense 3.6 27B?
Yeah, Qwen3.5-122B-A10B-NVFP4 produces better responses than Qwen3.6-27B-NVFP4 (both from unsloth), but I'm mostly using them for programming in various ways, mostly Rust, Clojure, Python and JavaScript, and some translations tasks, but not much more than that, so YMMV.
Edit: as a concrete example, I'm working on a "optimization framework via agent harness" right now, Qwen3.6-27B-NVFP4 is often unable to actually complete the optimization within 100 turns, while Qwen3.5-122B-A10B-NVFP4 has no issues finishing within ~50 turns or so.
> Yeah, Qwen3.5-122B-A10B-NVFP4 produces better responses than Qwen3.6-27B-NVFP4 (both from unsloth), but I'm mostly using them for programming in various ways, mostly Rust, Clojure, Python and JavaScript, and some translations tasks, but not much more than that, so YMMV. > > Edit: as a concrete example, I'm working on a "optimization framework via agent harness" right now, Qwen3.6-27B-NVFP4 is often unable to actually complete the optimization within 100 turns, while Qwen3.5-122B-A10B-NVFP4 has no issues finishing within ~50 turns or so.
But why are you using Qwen3.6-27B-NVFP4 compared to the FP8 or full version? In my experience the Q8 of 27B is on par sometimes better than 122B. I am experiemnting witb higher quants for 122B to fit on my Strix Halo, but still, the difference honestly for my workflow is not that much. I just wish they released 3.6-122B version.
dense small models do not like quantization. i find 27b fp8 to be smarter albeit less knowledgable versus the 122B
Hahaha well played! Hat tip
SO MUCH THIS. I have Strix Halo with 128GB RAM and was a large and fast model like 122B A10B. Here's hoping!
Absolutely. There's a glaring gap in the space for something about the size of Nemotron Super or just under, but actually ... competent.
The fantasy is a 100B or 80B model, but MoE and highly tuned for coding.
Indeed! That would be the sweet spot for my 2x3090 rig
This seems more of a battle for frontier AI supremacy. I'm afraid that small capable models have been left in the dust. Big labs don't really want to hand over the golden eggs goose to the end user. Possibly the hardware vendors(e.g. Nvidia) may want to play in that area as well, to pull money from all parties.
> I'm afraid that small capable models have been left in the dust
I wholly disagree. Rather than going the "everything is a claude code skill" route, I've been hacking together purpose-built harnesses for all sorts of tasks, and in that environment a wee little baby model can do some really useful things. You end up burning lots of tokens making the thing, but then all that investment comes back when the resulting tool works perfectly fine on a dinky little model that fits on my 3060 Ti.
Google makes Gemma 4 31B QAT; that's not a small lab. It's one of the better models out there for consumer hardware. Allows me to run it on a 7900XTX with 64k context.
nvidia and amd don't give a flying fuck about end users right now while they can milk triple digit markups from infinite money VCs via data center GPUs.
someone will keep putting out consumer level models. Once you have the larger models, you can derive the smaller onces.
Europe will definitely be interested in democratizing these things if China starts losing interests; from there, there'll be more countries looking to keep their citizens entrained in their own Country's infrastructure.
It'll especially be true if the memory cartel keeps prices high and NVIDIA tries to gouge higher memory models.
It's an arms race everyone can join because PC hardware was mostly democratized in the last decade.
That's a 2.4T model, how would they reduce this to 35B and still give some accuracy? That's a completely different arch.
there's been a lot of research about reducing models by taking out layers; there's also using it to train smaller models by optimizing parameters.
I dont see most model building as anything more than a pig at a slop troth, despite the level of sophistication; they're still rarely pruning the input beyond random sampling.
What hardware do you have?
I run a 2x 3090 rig, but a single 3090 already provides a great experience at a reasonable quant and context size. On a single card I used to run Qwen_Qwen3.6-27B-Q4_K_M or similarly quantized 35B MoE at 65536 context size
you should look into https://github.com/noonghunna/club-3090
For those trying to get it to work in OpenCode with a Qwen Cloud Token Plan, this is what worked for me. Note that I've just matched Qwen 3.7 Max for the limits as I don't know exactly what they are.
also, be very careful which API endpoint and API Token you use. make sure you use right one (obseve your quota is used up. if you hit right endpoint quota used almost immediately). so that you do not accidentally burn API endpoint tokens (they are expensive, can easily hit 200 USD / 3 days which do not count towards your membership "Credits", if you say purchased it with 200 USD signup bonus in Alibaba Cloud)
Qwen has set an excellent track record for architecting and releasing open-weight models that consumer-grade devices can run. What is needed the most right now is something similar to Bonsai 27B, with a modest memory footprint, but faster and more capable. On-device models can make up for intelligence by being faster, thinking longer, or doing more quick iteration rounds.
> What is needed the most right now is something similar to Bonsai 27B, with a modest memory footpint, but faster and more capable
Yeah, that'd be neat, but that's not what this announcement is about at all:
> With a massive 2.4T parameters
True. It was more of an open letter, with hopes that the Qwen team sees the comments in this thread.
dont we all deem the ability to improve large models as the defacto capability to produce small ones?
I don't think so, they have different constraints and require different optimizations, being able to produce one of them doesn't mean you'll automagically be good at the other.
but that's denying the singularity boostrap theory. Which I don't agree with, but if you can't harness a large model to make a small model, then we're going to have problems brining about the singularity.
If you do think there's some magical singularity, how do you comport?
I’d like a “Bonsai 2.8T.” That is, something that is near the Fable/Sol/K3 class, but capable of running locally on consumer hardware.
At 1.5 bits per weight it'll still be over 500gb - that's still not running on consumer hardware.
Best case they release smaller models. 120b class of qwen 3.8 would be incredible - it fits on device for those serious about AI, but without millions of dollars in hardware for terabytes of VRAM
I can't really blame them that the biggest labs focused on trainig and realeasing huge models.
The niche for small models should be filled with medium sized labs doing distillations of the huge ones into consumer grade hardware runnable models and LORAs for the huge ones.
I think AI will evolve the same way computers did. We're somewhere in the 80s-90s timeline of the evolution. My prediction is that on-device models will have excellent tool-calling, reasoning, and general skills, but the domain-specific knowledge will be retrieved on-demand from vendors like Google. Rather than downloading models, each device will have a hardware component with weights baked into silicon for maximum efficiency.
Weigths directly in silicon is a bad idea with the way the space is pacing. Just look at chatjimmy.ai it is fast, but on the once good llama3.1-8b but now pretty useless.
Qwen is the most censored of the Chinese models in my testing, which makes me wonder in what other ways it is compromised. Open weights doesn't really reveal what's in there. And, in my tests, existing Qwen models are not at the pareto frontier of any metric; DeepSeek V4 Pro is better, faster, and much cheaper than Qwen 3.7 Max. (DeepSeek is also among the least censored of the Chinese models.)
I guess we'll see if the "second only to Fable" hype pans out. In my limited experience with Kimi K3 (I signed up for a month of the $19 plan) it's slower and chews a lot more, so ends up being pretty expensive; one little feature burned through almost the entirety of my five hour limit. The $20 GPT plan is a lot more useful and includes 5.6 Sol, which is fast and token-efficient enough to be quite usable even with the small plan.
What censorship? ;) https://github.com/p-e-w/heretic
What's been your experience using these models? In my experiments, while it is true that the models are less likely to outright refuse to answer "sensitive" questions, they are still very resistant to actually respond in a meaningful / useful way.
You still don't know what's going on in there.
DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.
Deepseek is one of the worst in terms of hallucination rate according to artificial analysis' benchmark: https://artificialanalysis.ai/?omniscience=omniscience-hallu...
That's interesting. The "best" current models, Fable and especially GPT 5.6, are also lying liars that lie all the time. Seems like we're going the wrong way on hallucination.
And I remember Sam Altman saying 2 years ago or more, that hallucination was already "fixed" internally. Well, clearly not.
I've been seeing more "grounding" messages in the thinking log of Claude models. I guess that's "do a web search, read the docs", probably designed to mitigate hallucinations.
I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.
> I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.
In my mostly unscientific RAG experiments I found that larger models were more likely to hallucinate in a RAG setting. I think that it's because they have more world knowledge. As an example, when asking about safety legislation in Ireland, Claude got hooked on the notion that OSHA was involved, which clearly is not true, while the smaller models just said that they couldn't find any information (which was the correct answer).
Deepseek V4 pro is a heavily undertrained model, they only trained it a bit more than the small version and that small version is 6-ish times smaller. Ive found that Flash is absolutely incredible as a workhorse for wide scale agentic nonsense, but Pro is a bit undercooked and really goes on strange tangents very often.
I have seen occasional weird behavior that I guess could be attributed to hallucinations, but for security auditing, DeepSeek v4 Pro is among the best models I've tested, competitive with Opus 4.8 and GPT 5.5 (MiMo and GLM also did well, Qwen 3.7 Max was below all of those, though only barely), and at an order of magnitude lower cost per task.
The nice thing about it being open-weight is that you can uncensor it.
I asked it similar questions of Chinese human rights and it started with “your premise is incorrect bla bla bla” and then it just redacted the whole thing and showed me an error code.
Try it yourself here: https://www.qwencloud.com/try-ai/chat
I had a fun conversation with Qwen 3.5 a while back. I ended up getting it to admit that it was complicit in the coverup of the Tiananmen Square massacre. This was running locally—it wouldn't surprise me if they had additional safeguards for their hosted service.
“You can't trust Melanie, but you can trust Melanie to be Melanie.”
I'm haven't found an announcement page, but there's a banner on the website announcing Qwen 3.8 and redirecting to this page.
Looks like they're previewing the model only on their subscription plan.
(This comment was originally on another merged post, and "this page" referred to https://www.qwencloud.com/pricing/token-plan)
It’s available in the iOS app (or was for me), both logged in and out.
Is there an iOS app for using Qwen?!
Yes, but it’s not available in all App Store regions.
Things are heating up in China.
Looking forward to see what Antrophic and OpenAI does next.
OpenAntrophic and OpenOpenAI ?
Over Dario's dead body.
At least Sam have us a nice model last year with the gpt-oss family.
I really like Qwen, even the Q2KP quants of 3.6 27B have genuinely impressive local performance on a 24GB card. It has been good enough that I am happily giving them $60 right now to try this instead of waiting to try a slightly lesser version locally.
Was there ever an explanation for why we never got the weights of 3.7? I would like sourced quotes and not weird/cringe accusative speculation about distillation, or your take on The Big D.
do you mind sharing your settings? i just picked up an r9700 to start playing with local qwen3.6 27b and your setup sounds promising and efficient on 24gb.
Which settings exactly do you want?
As far as the model settings go I just follow what's on the card.
I'm using HauhauCS's models, they seem to do a slightly better job with their "P" quants. Especially wrt patching them to eliminate the "doom loops" that will time out the GPU (esp if you have not already given it the extra power budget, set fans to max, and lowered your max clock by about 8% to save yourself a crash/reboot).
Without getting into the weeds, unsloth covers a lot more ground so, when they're good they're great, but I've also had the most problems with them. Basically, don't be afraid to shop around and fuck with sliders.
These days it should almost always be enough to open up LMStudio, set context to max, K/V quant to 16 or 8, and off you go. I'm using a 7900XTX, with 128GB of memory for the cases where things don't fit. The default settings should be fine otherwise.
I don't know anything about the r9700, seems neat. Looks like it might play nicer with the vulkan backend than rocm, and you might have to `set GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1`. Perf should be at least as good as what I'm getting (180+ tk/s prompt, >50tk/s response), which imho is just fast enough that I can read it as it reasons and responds.
You can probably get more aggressive with KV quant (any of the 4's) and bump up the batch sizes (4096/1024 vs 2048/512, etc), it's quite situational. These days there generally should not be any serious loss of accuracy from the former. Keeping the rig cool will likely matter more, so turn your music up to hide the fans and set your rig on top of the AC exhaust lol. Oh and I guess make sure ReBAR is enabled and working, adrenaline should let you know if it's not enabled.
Different quants, but I've been having success with: https://unsloth.ai/docs/models/qwen3.6 I do llama.cpp
The "second only to Fable 5" comment is pretty telling here. I remember early on when a lot of naysayers were saying that Fable was barely an improvement on Opus. Like it or not, Anthropic have a genuine moat right now with that model, provided they continue to allow people to use it. It will be genuinely exciting when an open model is able to beat it.
I wouldn't call it a moat, but I would call it a noticeably better model. Subjectively, for my own work, I would rate the top models Fable > K3 > Sol.
But it's not like Fable is so substantially better than the other two that I would be seriously impacted if I didn't have access to it anymore. All three are amazing models, and of the three, Fable is the only one that regularly triggers refusals.
It really does depend on your application. In my domain (math research), it is substantially better. Fable can solve really hard tasks with surprising consistency. It makes mistakes, and occasionally refuses, but honestly, at the top level, ideas are the currency and the rigor is the busywork. The other models cannot come close in this domain.
If you couple Fable's idea factory with Sol's rigor, you get a real game-changer. It puts the emphasis on top-level ideas, and nearly trivialises the intermediate layers.
It's always fun to see what works for others, because for my work it'd have to be Sol > Fable. Fable makes too many mistakes.
Coordinating agents though? Fable any day.
tbh even if its better model due to lot of restrictions its not that useful than opus.
100% this. There's currently this [1] submission that hasn't gained much attention, but is really important. In this [2] incident report from HuggingFace, they talk about detecting an attack and not being able to analyse the logs / IoC with API models because of guardrails. If not even highly regarded reputable companies can't sort out access to SotA models for blue team use, the raw capabilities don't matter. They're useless paperweights (hah!), and nothing else. Having to resort to open models is insane!
[1] - https://news.ycombinator.com/item?id=48965243
[2] - https://huggingface.co/blog/security-incident-july-2026
Key part from [2]:
"When we started the log analysis, we first used frontier models behind commercial APIs. This did not work [...] We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. [...] The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout [...]"
Tell that to my colleagues. Despite Sol getting the attention, Fable is really starting to have an impact on mathematicians right now. It has unbelievable insights in a lot of cases that can rapidly speed up progress.
their "genuine moat" is that mythos is the only super heavyweight model right now. it's always been possible to train a 10T model and get 10% more performance over a 1T model.
had mythos been just Opus 5, with the same size and price as the previous opuses, then yeah, that would be a tie-breaker. but it's not.
I dunno, I find Fable slops alot. Sol is my workhorse. Fable can be creative but isn't very good at doing work reliably (or without endlessly burning tokens).
I am not even sure if Fable is as smart as they say, I can't get it to answer almost any question, it always refuses for "cyber-security" concerns...
5.6-sol would be a better comparison given it's general availability and usage allowances
They are likely assessing based on "raw intelligence" benchmarks, rather than agentic ones. Fable crushes in those, but that doesn't necessarily translate to microscopic rigor, which is what most people use these models for. You only see it when you ask really tough questions.
And sol is much more reliable as a agent for doing work. Fable sometimes just goes on wild flights of fancy.
More like a puddle than a moat
Fable is still as dumb as a post. I ask it simple questions and it routinely gets things backwards, prioritises things that should be subordinate to others, etc.
An example: It just suggested that I shouldn't raise the price of my saas because it'd complicate the arithmetic if I did 0 -> $100k YT channel instead of sticking to $20 p/m.
It's just a complete moron, like all of them.
> saying that Fable was barely an improvement on Opus. Like it or not, Anthropic have a genuine moat right now with that model,
What's more interesting is that Anthropic moat shrunk to just that model. There's zero reason to use any other model from Anthropic right now. And once they take Fable off subscription there will be zero reason to have Anthropic subscription.
That’s not a moat though
What's the point of Fable if we can't use it? I get to prompt it like 5 times on my subscription before it gets cut off, and even then I'm constantly fighting the insufferable safety classifier.
I'll switch to OpenAI soon because of this. I also can't wait for the day it becomes feasible to run these awesome open weight models on my own hardware.
> You don't have to wait to test it. Just now, the Qwen3.8-Max-Preview made its debut on Alibaba’s Token Plan, Qoder, and QoderWork. Be among the very first to try it out.
You can also try it out on Qwen chat, Its free.
> "compatible" instead of "comparable."
It boggles my mind how you can train a frontier model but not write a tweet without an obvious typo.
at the risk of upsetting a lot of people mentioning this but like are you really surprised?
if youre going to use ai for everything youre gonna start losing your edge as you focus less and less on what youre doing and this isnt me just talking out my ass, like... the front page here is peppered with study after study and blogpost after blogpost about how its overuse can come to the detriment of one's own abilities and skills.
coca cola had the ad with the magical truck that changed its design, shape and amount of tires it had and if nobody noticed that before releasing it then im not sure why anyone might think that the people peddling the LLMs would somehow be immune to this phenomenon
This is the sort of unmitigated pedantry thats the real problem. Everyone makes typos and errors of this nature. Feymann and Hemmingway both did. You deliberately chose this level of error multiple times when you chose to not put an apostrophe in youre and failed to capitalize proper nouns
Come off that high horse
i literally have no idea what you're talking about my dude/gal and/or pal.
i just dont have autocorrect turned on; chill.
The people training the model are almost certainly not the same ones writing tweets. I don't know why that is mind boggling. We all make typos, at least the humans among us do.
let's see your grammatical correct tweet in chinese.
Well, I think Chinese doesn't really have any grammar most of the time.
Now you can be sure they write their announcements by hand. Doesn't really matter, does it?
It's OK because I'm not prompting the person who tweeted for my usecases.
mistakes are at the moment a signature quality of human written texts, enjoy them
i'll take that 100% over llm slop.
> With a massive 2.4T parameters, this model is continuously evolving. We believe it’s one of the most powerful model available today, compatible to leading frontier AI models , second only to Fable 5.
That's a massive model!
The shift from "value" models to "intelligent, huge and slow" models coming from China is an interesting change in strategy.
My main issue with GLM 5.2 and Kimi 3 is that they're extremely token hungry and thus feel slow(er) to use.
Value models are always going to be there; you can always distill from a larger model. Having a really intelligent model, regardless of the size, is much better for building confidence in your brand. That is a big reason why the US companies are still hanging in there.
The shift isn't new. Kimi K2, a 1T model came out July last year. I am happy that more labs are following the trend as its important for competitive open models to exist.
Also DeepSeek R1 was announced 1.5 years ago with ~0.7T parameters, which was a huge model back then.
And DeepSeek has been making huge progress on efficiency, and publishing about, so they came with a 1.6T model that is both fast and cheap to run.
The title is misleading. The link led to a pricing page/token plan and not about the new QWen 3.8 model.
The submission link is to the twitter announcement. The body just has a different link to pricing.
I remember when they released Qwen 3.7 Plus and Max. These models behaved way different from all prior models, it became too verbose. It wrote multiple paragraphs just to answer my prompt instead of the usual concise and direct way responding to me. I didn't like that at all, and I know Gemini also had this behaviour with with the Flash series until I managed to reduce it a bit with personal instructions (in the settings on Gemini website).
I haven't tried Qwen 3.8 Max yet, looking forward to it. My hope is that its way less verbose. Another thing I experience with the Qwen models is that I do not trust their benchmark scores at all. Have anyone played with Qwen 3.8 Max and can share their experience? Which model it come close to? Sonnet 5? GLm-5? DS V4 Pro? Flash? Gemini 3.5 Flash?
Now there are not one, but two incredibly powerful open LLMs. I think this level of capability makes general prioritization / high level decision making doable with the right harness, and now everyone has hard-to-interrupt access to them (since these are open weights and someone in the world is going to run them). This world is going to get really weird soon, in both good and bad ways...
Let’s better download them fast before Dario is making a scene again!
If he bans them, they'll just show up as torrents. Or we'll download them from Chinese hugging face.
I do hope they provide optimized A3B quants--that's been the sweet spot for usable local inference for me, at least.
I did some comparisons to Kimi K3... and found the best is combining the two! Model diversity is powerful.
https://trilogyai.substack.com/p/qwen-38-max-benchmark-how-i...
What is a "credit" and how does it translate to tokens for the different models?
Its the currency of the future.
Since Euro and Dollar values are reasonably close, you can call both of them credits, maybe
Yes, but credits are money you don’t actually own. So much more convenient, wave of the future and all that.
They use it in Babylon5 (supposedly) in 2260!
Wait.. the Qwen Max models have never been open-weight. But it sure sound like that's what they intend now?
"Qwen3.8 is launching and going open-weight soon! With a massive 2.4T parameters..."
Why did Qwen stop producing open models? They've gone from building the best open models ~1 year ago to producing like the 10th-best closed models. I don't understand this pivot at all.
Edit: I saw online they do in fact plan to release this openly at some point – x.com/Alibaba_Qwen/status/2078759124914098291
They've announced that they're releasing the weights for a 2.4T model soon:
https://xcancel.com/Alibaba_Qwen/status/2078759124914098291
https://news.ycombinator.com/item?id=48966120
It's not a pivot, giving away the weights was a marketing strategy that they don't need to keep up with.
Always nice to see more open-weights in the heavy model class. I can only hope this trend continues, causing OpenAI and Anthropic to crash and burn.
As much as I dislike 'em, this sounds mean spirited. And Alibaba admits in this very tweet that Fable is next level (it is).
I dislike their practices, but the main motivation for hoping they'll crash is that I think their immense overvaluation posses too much economic risk.
I want as much misfortune as possible to befall OpenAI and Sam Altman after what they did to the memory market.
How dare they buy things
But they didn't. The deal to buy 40% of the world's memory never happened after the price increases the news generated.
How dare they not buy things
How dare they claim to buy stuff to boost their standing while actually just faking the entire thing.
Same, you can think of China whatever you want but they're really good at giving big tech a reality check when it comes to AI. We now got a pretty wide range of open-weight models (from DeepSeek and MiMo over to Kimi K3, Qwen 3.8 & GLM-5.2) and I think it's most important that there's a variance not only between quality / intelligence and also cost.
I mean even the cheapest option for Luna is still more expensive than anything DS or MiMo is offering right now and I think a new Ministral model would also hit hard there because we also need some variance in model sources, we can't rely only on the US and China.
"China" isn't giving anything. These are Chinese companies leveraging their best competitive strategy at the moment: competing on price.
Those Chinese companies are being funded by a substantial amount of government financing, I think at least 20% has come directly from state owned investment firms or government entities and probably more now. You obviously lose precision when you’re talking in sweeping terms like “China” but I don’t think it’s entirely unreasonable in this case. The Chinese government is playing a much more direct role in AI investment and research than other nations.
Nonetheless, the government isn't giving it away either. If China had a monopoly on the technology, or simply winning in quality, open weight models would never have seen the light of the day.
I just want to dispell the silly notion of altruism from China in this conversation.
Bad chyna
The Chinese government is explicitly calling for open weight AI models, which influence its companies.
https://news.ycombinator.com/item?id=48970449
Damaging American AI companies just means that China has the lead with closed models.
China isn’t an altruistic state. They’re an aggressor in many fields, economic and otherwise, and this one of them.
Has there been any news on open weighting Qwen Image 2.0 and WAN?
We are spoiled in the LLM segment, but I would love to see an open source competitor to Flux.2, etc.
Does anyone have the privacy policy of their token plan available? Want to check if they retain/train on inputs/outputs.
Lol, lmao even.
Of course they train on literally everything they get their hands on, like everyone else. If you need privacy, that's what local models are for.
It's a flippant answer to a real question. Anthropic, OpenAI, and even Grok have "Don't train on my data" knobs.
Whether you trust them is different, but there ARE knobs on other hosted AI companies.
> Anthropic, OpenAI, and even Grok have "Don't train on my data" knobs.
Would you trust this?
Anthropic paid 1.5 billion for illegally training on stuff and that's even without a judge forcing them to do this. This seems to be their modus operandi
Those knobs don't do anything, don't be silly. It's just optics.
If this is true then why did Anthropic bother to update their privacy policy when they first launched Fable?
I don't have the energy to cynically squint at everything in my life. If a company says "We won't track you" I'll believe it until proven otherwise.
When these companies prove dishonest, I'll adopt skepticism.
Do you have proof of that?
Do you have proof it's not? There are no laws they have to follow in regards to it, and it's practically impossible for them to go against their core self interest. More data is literally a direct component to better models and more revenue. If someone proves it's fake worst they get is a month of bad PR and then people will move on, as they always do.
> If someone proves it's fake worst they get is a month of bad PR and then people will move on.
You can say they stole from everyone to train their models in the first place and that's valid, but this isn't that. You are saying they are actively ex-filtrating data from any company using their services and lying about it.
Google/Apple/Microsoft or all of the dozen trillion dollar companies in the US would absolutely crush them in litigation. Neither OpenAI or Anthropic would be able to survive it. It's just not worth the risk.
I'm sure in the handful of cases where they have a genuine bulk contract on paper with a large firm that states they'll be in breach if they train on it, they do abide by it.
But unless you are one of those you mentioned and a few others you probably aren't notable enough to care about. Everyone who uses their services directly, paying or not, is surely ignored in that sense. I wouldn't be surprised if there's a team of their own lawyers ready to interpret their EULA in fascinating ways.
And out of those three I'd only probably assume Apple is the only one who doesn't use the data given that they've built up privacy as a selling point, MS and Google probably train their own models on it themselves.
The few tests I ran were by no means comprehensive, but while kimi felt like the real deal qwen seems a bit of a benchmark princess.
Qwen3.6 is still the best agentic open weight LLM around 30b params (Gemma isn’t very good at agentic execution).
I also find the model is a lot more predictable and less “glitchy” when made to think in Chinese. You can do this in the system prompt.
> Gemma isn’t very good at agentic execution
I had no issues with it for C++ development with https://pi.dev. I'm yet to try it with Zed Editor. I don't rely on agents too much. However, I used it on Chromium's codebase to research some functionalities, let's say for searching. Requests like: check my last commit and do the same for SetterA and SetterB; it also ran without any errors.
Apparently agentic performance in Gemma was improved recently: https://x.com/googlegemma/status/2077449152062247219
Too little too late imo
Open Source Era is comming
Looking forward for a 20ish billion parameter version
Qwen is so much better than GLM or Kimi that this makes me genuinely excited!!
The bigger models are usually worse though, hopefully they nail it this time.
Does anyone know if they intend on releasing open source/weights variants for 3.8 or whether 3.6 was the last model they are/were doing that for?
will be releasing weights per their tweet announcing the model https://xcancel.com/Alibaba_Qwen/status/2078759124914098291
The only problem I had with Qwen, fine tuning on Colab, it takes 31 t/s while Gemma 4 is around 9 t/s, otherwise, one of best local LLM
With 3.6 27b, I just stopped changing local models and started tinkering with things on top (like mem0). Feels genuinely useful and more than a toy
I have only been using 3.6 27B for coding. Is mem0 for agents like Openclaw or Hermes? How are you using it?
It's a mcp, so connects quite easily to agents. With mcpo, I also connected it to open-webui (which has better support for OpenAI style tools/functions). Used it in claude code with that mcp plugin set-up too. Only ever used it for managing homelab information, but it met initial expectations. 27b is a great model, if grounded. The query about physical hosts and routing... I haven't found a single hallucination (altough Codex 5.6 as a reviewer mentioned something was wrong with some parts, and those were exactly the never properly documented ones. Codex/gpt had extra knowledge, because it was the conversation I used to set it up).
These models are great, but what's the potential use? No small entity can run them.
China encourages/prioritizes their release because it directly competes with American companies closed models.
If 3.8 max goes open weight, what are the odds they retroactively open weight the earlier releases?
What would be the point? Kimi is kind of forcing their hand.
Well, if nothing else, posterity.
It feels like an inflection point of lost US leadership in technology? A year plus ago you would say while China led in green energy and manufacturing, at least the US was ahead in software - as demonstrated by the state of US AI models.
We could point at a lot of factors on the US side. From political paralysis / head-in-the-sand attitudes towards emerging tech like green energy. To something of disdain for workers that will be impacted by AI (creating a backlash). To education that continues to lag. Add to this so many other self-inflicted economic wounds from the current administration.
I don't know if its nearly as terminal, as say the UK after WW2. The US is still large, wealthy, and resource rich. Yet at a minimum the triumphalism about US leadership after Trump was elected by the tech elite feels silly in retrospect.
Something I also think about is how much stronger The West overall would be if instead of antagonizing allies, there was a single ecosystem working closer together.
> We could point at a lot of factors on the US side.
It's just lack of antitrust enforcement.
China pours money into tons of different businesses in the same industry and lets them fight it out. The only US business model left now is to shut down (or collude with) competitors and raise prices while cutting costs. All they have to do is cut Congress (and regulators, and individual judges) in. We've financialized everything for the sake of scammers, rather that finance being used for the sake of getting cash to the most productive organizations. We've optimized for corruption.
If we hadn't let the stupidest people in the world buy up everything, and made doing nothing with it the most profitable option, China would have never have blown past us.
The US Supreme Court has explicitly legalized "tipping" politicians. That's the biggest sign of degeneracy that a government could possibly achieve.
https://en.wikipedia.org/wiki/Snyder_v._United_States
in case someone wants to speculate in why chinese labs open source their models: https://try.works/why-chinese-ai-labs-went-open-and-will-rem...
If it offers more than opencode go, the entry plan looks enticing
I predict that no one will use this and everyone will use Kimi K3.
I've been playing around with K3 a bunch, but the verbosity of the reasoning makes complete e2e agent work basically cost the same as other smaller models, and I'm not seeing a huge difference in quality, just a way longer e2e completion time.
Same problem with every chinese model currently, they overthink way too much and take too much tokens and time.
More or less, yeah. I've found mild success with deepseek-v4-flash though, and also Qwen3.5-122B-A10B-NVFP4 running locally, especially in terms of "doesn't overthink every single prompt" and somewhat reasonable quality. Really wishing for a 3.8 update of the 122B variant, that'd be really competitive (for local usage) :)
A consequence of aggressive distillation?
Why? Price? If the reason is performance, I've been using non-frontier models for cheap, and they run great for my needs (GLM 5.2, DeepSeek v4 Pro).
We'll probably use it, but for images. Qwen is still the best for images
What's the price difference?
Who is behind this site? Is this another frontend to Alibaba or a reseller in Singapore?
Go China, screw America*
*within the scope of open models only
I like my Apache 2.0 licensed Gemma, and NVIDIA’s Nemotrons are decent bases for finetuning or continued pretraining, esp thanks to good documentation and tooling.
Oh, and Mira’s thinking machines lab dropped Inkling, a ~1T open weight model too.
This isn’t US vs China. This is open vs closed.
This isn’t open though. Promises to be open later aren’t worth anything, given what we’ve seen and heard from AI execs making promises in this industry.
it takes extra effort to open source a model even if you had it running internally. even traditional software takes extra effort to get released as open source.
I’m sympathetic but AI claims are to be disbelieved until proven anymore. That’s a well poisoned thoroughly by not just American companies.
It's Sunday morning so I'm allowed to be facetious!
can i get "code instruct" version of this? i want 7B and 14B to launch on my hardware.
interesting, when will this race end?
This is reminiscent of the operating system wars and browser wars. In the end there will only be 2 models that can survive. 1: give it away for free or 2: locked in with top notch hardware.
I very much appreciate the existence of these free models, but in my experience Qwen has too high of a tendency to confidently give the wrong answer as compared to other frontier models.
So are locally-runnable models frozen at Qwen 3.6 now :/
Everyone wanted open models that would challenge Opus and Codex, here, you got it.
We need better coding models that can run on local hardware, i.e. 128GB VRAM or less
You can run larger models by offloading to SSD (for weights), it's just slow so people don't do it all that much. But you can get back at least some of that performance by using either MTP (at least for dense models; not effective for sparse MoE models unless you're batching them already and have VASTLY more parallel compute than you'd know what to do with) or batching multiple requests in parallel (note, this hurts throughput for your single sessions but running more sessions in parallel still boosts your total amount of inference. This requires careful management of memory requirements for your context/KV cache, and Qwen models tend to be KV-cache heavy).
Broadly speaking, this ultimately pushes local inference towards a challenging world where you use SSD offload for weights as a matter of course; then smaller requests (or requests sharing the bulk of their context, e.g. subagent swarms) can be batched together and run quickly in aggregate, but running very large contexts will actually limit you to single-session inference and require swapping out even the KV cache itself to some external scratch SSD, further hurting your performance. Then feel free to add wide use of MTP in a probably futile effort to go back to tolerable tok/s numbers.
Queen has that already, although they seem to be moving away from local models unfortunately.
Is qwen 3.6 27b the best model you can run locally at the moment? Not that I have the VRAM for it, but just curious.
In my experience, yes. A bit more reliable than gemma for me. I mostly use A3B (35B, mix of experts) though, because it‘s faster, and in the same ballpark intelligence wise as the dense 27B, so it’s the sweetspot for me. I want to try cohere‘s mini code model next, but worried the runtimes aren‘t optimized for that yet.
I found qwen3.6:26b slightly better on my 32G mac mini than the same sized gemma until gemma was updated with better tool support 4 or 5 days ago.
It is like a ping-pong game: the advantage flips back and forth between providers.
Worth knowing that Unsloth have just put out another Gemma 4 release from Google's upstream updates which should improve reliability. Bugs in the chat template affecting tool calling and other issues, apparently. https://www.reddit.com/r/unsloth/s/MpC6Hzs4Wj
Wow, thanks. I didn't see Unsloth had already done their version; I was just about to go back to the google version to test this change.
Yeah, it's only a chat template change. So if you don't fancy re-downloading the whole model and can just hack the new chat template into your process, it's a lightweight test to do.
I have been running 3.6 27b on a dual AMD r9700 setup using Opencode and Matt Pocock's skills workflow for writing Golang CLIs. It's decent, but won't win any awards on code architecture. I guess you can try to AGENTS.md the deficits but I am just exploring its raw Opencode experience right now. Much slower than an API but still 3x times faster than I can read. Tuning it in with a community chat template and a specific penalty for repeats was the sauce needed to get it to work. I can probably start loop daddying it now over the tickets Matt's flow creates.
So yeah, it's the best local model I've seen. I am going to try the Qwopus 3.6 fine tune soon with the same spec and tickets and compare the output of both.
Would you mind sharing more please? I literally just finished the same set up, with a 9950X CPU and 192G RAM at 4,800 MT/s. I used lemonade with Vulkan and the UD-Q8_x_x model from HF. 256k context. I have about 8G VRAM free, and use the iGPU for my desktop/monitor on Arch. What options do you give llama-cpp or whatever you run please? What other models have you found fit nicely in the 2xR9700? Thanks!!
I actually have long discussions with Gemini about this and have wound up download a bunch of different models for different things. There is no best, just fast but worse, slow but better, agentic or not, reasoning or not great at large contexts, better world knowledge, uncensored, etc…. It’s a bit daunting actually since there isn’t really a one size fits all model that you can just use for everything.
Yes it's between this and Gemma 4 31B which is much slower, but looks like it won't ever get an upgrade. I have to conclude that the MoE variants are unreliable, and MTP sometimes just can't get tricky formatting right.
The whole series had an upgrade a couple of days ago actually — they have addressed embedded tool calling (and hopefully the MTP formatting stuff though I gave up running the Gemma MTP because it's often slower than not-MTP)
Not tried it yet but I've seen tests that suggest they've properly fixed the tool calling issues.
I find the 4-bit QAT with MTP to be entirely usable speed on both my boxes (Strix Halo and a desktop with two V620 GPUs, which are slightly faster than the Strix Halo).
For whatever reason prefill (on my DGX Spark) is faster with the Gemma models than Qwen 3.6 models of similar size. On vLLM anyways. Likely just deeply tuned code contributed to vLLM by Google?
vLLM gives me ~7000+ tok/sec with Gemma 4's MoE model. Vs ~6000 tok/sec for Qwen 3.6 MoE.
People have been able to run DeepSeek v4 flash with a high spec Mac.
I flip flop between qwen 3.6 27b and qwen 3.6 35b 4b active.
But there’s also the quantization of DeepSeek v4 flash called dwarfstar
Gemma4 models are arguably better. Or at least about the same.
The best model you can run locally is Kimi K3, as long as you have the hardware. If "what model I can still run on a something resembling something I can put on desktop without separate electricity and cooling water inputs", then it is probably GLM 5.2 (can be run on e.g. Nvidia DGX Station workstation). As long as you have about $100k-$150k.
Maybe, maybe not. Qwen 3.6 27B is literally just three months old. Hard to predict. Maybe it just wasn't worth making a 3.7, and after all, the 27B release was after the Plus release.
The open weights vs frontier models is reminding me more and more of the Linux vs Windows I grew up with (slashdot randomly popped into my head saying that)
I have a feeling this is the next…frontier of that fight
One can only hope it eventually does as well as Linux
Obligatory "Does it answer security questions?".
It's an interesting time to be alive when your local models are supposedly the pinnacle of what a free nation is capable of, yet the ethicality of the companies is disliked and their models restrict and limit you so much you root for the models from a socialist/communist state. If it wasn't for the effectiveness of propaganda, tribalism and psyops in this scenario my words wouldn't even be controversial and would just be seen as a truthful observation.
nice
waic go brr
Nice
Open what?
Do this giant open-weight models have less active params and could be run on consumer hardware or no ?
Any open-weights model that has ever been published can be run on consumer hardware, even on a mini-PC.
The right question is which is the speed that can be achieved on a given hardware and whether it is high enough for the model to be useful.
Until now, the speeds reported for running big LLMs with the weights stored on SSDs have ranged from as low as a token every 10 seconds or so, to as high as a few tokens per second.
With open weights models that you host yourself, you are not constrained to use any single model, because that is the one for which you pay a subscription.
You can use many models, each for whatever it is more suitable. You can use frequently a small model with a high inference speed, but for some tasks you may actually save time with a better model, even if it is much slower.
In my opinion, even at 1 token per second a big model may be useful for some tasks.
I can only imagine what that does to the poor SSD
Paging in experts is mostly reading so on the first order effects it would be fine. These might be second order effects from e.g. write caches needing to be flushed more often (and maybe swapping other applications, if you use a swap file or partition) but it probably wouldn't be too much of an issue.
SVG's pelican https://gist.github.com/vitordelucca/521c2d63c9b852c622e7648...
Made on the website, so not sure if on the API there's more thinking options...
I feel like the pelican test can't be relevant anymore; the whole point was to to something that wouldn't be in the training set at all and now it is?
A parachuting flamingo? An aardvark driving a bus? It should be easy to randomize the animal and the mode of transport (or vary it with animal playing a sport) to create images not in the training data.
How about an animated SVG of a pelican doing the Macarena, profile view, spinning to face the camera on the last beats?
Agree. It was interesting/fun for a bit though.
What about an armadillo playing a piano? There are so many potential combinations It would say something if the pelican looked great but the armadillo looked terrible
> I feel like the pelican test can't be relevant anymore;
It never was. The point of this "pelican test" was for performative reasons, or just for attention of the joke.
It is like trying to test whether if an adult elephant could actually climb up a tree and reporting that some elephants are slightly better at doing that than others while also reporting at the same time that they are all bad at tree climbing anyway.
This is an example of testing for the sake of testing. The "pelican test" tests for nothing.
Using QWEN models since 2.5. I never used the chat properly but as an API I can say they're quite good, especially when you compare with OpenAI models. Cheaper and almost same level. I will try this now also.
Just imagine Anthropic making Opus open-weights now for the sake of trolling everyone. Wouldn't surprise me at this point xD
That would go against everything that Dario believes in (note that I refer to the CEO and not the company; the staff at Anthropic are not so ridiculous). He believes in Anthropic being the sole arbiter of the forefront of this technology, because it is all too dangerous in the hands of anyone else.
I’ve seen no evidence that he believes in anything. He comes off as just another slimy would-be monopolist to me.
I see no evidence that any ceo retains anything but the desire to capitalize on their marketplace of ideas for their own benefit. Like wolves inn sheep clothing, they'll put on any skin suit to convince people to keep giving them money and power.
And it has nothing to do with the individual, from what I can tell, 70% of the population placed in their position would become the same type of uberpath.
That OpenAI releases Sol as downloadable weights feels way more likely than Anthropic releasing even the tiniest of models for download.
Has Anthropic released anything at all, ever?
Anthropic has a lot of interpretability work, but they are extremely defensive about everything else. Dario doesn't really believe in releasing anything. Despite how he acts in terms of some sort of highly principle driven saviour (machines of loving grace) he clearly is more business minded then anything else.
For example they don't even tell you anything about the tokenization. They even do random chunking and padding to avoid leaking the token strings in the streaming api after it got reverse engineered. (See: https://spylab.ai/blog/claude-tokenizer/)
Not voluntarily
It amuses me that that PRs for the claude code source code are still openm
Model Context Protocol
They released it because they knew MCP was slop
safety datasets and a lot of safety related research