You're better off going directly to artificial analysis, this is a feature-poor/misleading/outdated repackaging
Ex. this type of price estimation is quite naive - some models can require 2-3x the number of tokens to achieve the same level of intelligence. Artificial Analysis' own cost per task is a more fair estimation of cost.
The analysis might be better on Artifical Analysis' however the way the information is presented by terrydjony is so much better. I'm able to get the exact actionable info I want. Whereas on artificial Analysis I'm analyzing and figuring it out.
It's the equivalent of ai slop writing in a data format.
If you have a 24-64GB mac, consider running Qwen3.8 27B locally at night. It's a bit slower to run locally, but if you're sleeping it's less of a problem.
Depending on your memory, you'll need to use the weaker Q4 versions but they still perform well.
It ranks higher than GPT-5.3 Codex (xhigh) or Claude Opus 4.6 (max) so is great for pairing with https://github.com/kunchenguid/gnhf for nightly experimentation, cleanup, or recommendation lists for in the morning.
I have all sorts of local compute, and local models fairly capable the Frontier models still way more capable/faster and local electricity consumption is something else. Good thing it is getting colder around here.
I often pair them up, and I have an Astra or Sol work as a supervisor and reviewer while Qwen 27B FP8 or Qwen 3.8 Flash Next implements things.
I mostly do it as an experiment, just to see what kind of level of autonomy I can get, and they are slow to getting a decent reviewed outcome despite Qwen27B running at 100+ tps and 3.5-4K prefill rates and Qwen3.8 Next at 40 tps and 1-2K prefill.
I've been also using similar approach more with OMP, not just the straight Pi harness. And OMP seems to be slower because it has more guardrails.
OMP has an interesting feature where you can assign a better LLM as an advisor, wehere it just sort of monitors the progress and injects guidance. And it definitely helps, but one has to be careful. It actually turns out to be expensive if the cache reads are expensive. I learned it the hard way. Where on Fireworks' API, the cache rates for GLM 5.3 flash are quite a bit more expensive than for DeepSeek, and a simple runs ended up costing me three bucks in oversight.
So a better way is to have a Frontier model running a separate tmux pane and just directing it to Wake up every 10 minutes, take a peek at what's going on, review the milestones give feedback and then sleep. This turns out to be pretty decent cost saving strategy when quote needs to be stretched.
Paradoxically, OpenAI tightening up their quota allowance once they released Astra actually pushed me into all these sorts of experiments, and it's actually been interesting. I've been exploring all these smaller flash models, and it's been nice.
I do like using local LLMs for chore type tasks that are just mostly information gathering, post-session reviews, stuff like that.
There also exists a $600-800 GPU that can run Qwen 27B 3.8 @ like 60 t/s for around 200W of energy.
I find Qwen Flash Next quite competent as well. 27B is a solid worker like you said. If you batch work and let them crank they do remarkably well. I am working on.
I started using Herdr and taught my agents to use it. So I use OMP loop and or goal, and it has a review cycles to wake up an Opus or Sol reviewer to make sure nothing is going off the rails with a local qwen flash next coordinating for me. I kinda prefer Sol, it seems like a more patient and thorough model, especially Sol 6, but Opus 5.5 is really good and its voice and attitude is not as grating as Opus 5 for sure.
I have a few V100 GPU running Qwen 27B and they do all the work overnight. Not quite the same speeds you have yet, but this is V100 machine and an old gaming machine with 16GB 4080 and a handful of 32GB V100s... all in less than 3K (ignoring that my gaming machine is 3 years old, but runs qwen flash next for free now as I game not a lot) for my little "we have AI at home" projects and there is a lot of interest in these old GPU now because they are rolling out of data centers now.
For my local work and personal projects... they just seem to be getting done in this setup. Every few days I sit down and do a big cycle with astra/fable/opus batch things up. I have projects that are basically "i want to see what happens" to "I want this to be good, I understand the code". Some of the throwaway projects that have just kind of magically finished more or less how I wanted have been great.
What do you mean by "local electricity consumption is something else"? Doesn't an M3 Pro for example draw about as much power as a bright incandescent lightbulb for a maxed out gpu workload (~100W)? That's less than a tenth of what a frontier model will use in the datacenter (which I believe are racks of BlackWell or Vera Lynn GPUs, each using 500W+).
Because M3 Pro is not a real thing as far as actual agentic workflows go.
I run dual R9700 boxes those idle at 150W Because of a Ryzen AM5, and I run Spark boxes, which are decent But still idle at 45-50 watts a pop. My favorite, to be honest, is a 5090 with a Qwen 27B because that one is good for quick hitters and flies, But again, the box itself idles at 140 watts.
So it's really the idle power that I don't like And it's too much of an inconvenience to power boxes down and power them back on, so they just end up running and sucking electricity. I am working on getting some sort of a smarter proxy setup where I would give boxes time to wake up and go to the cloud while they're waking up. So the whole point of having these things running in the background is that you do want them to be almost always available. So power consumption is definitely an issue.
for 32gb, this is my model of reference now, you have to run it with a patched version of llama and is still not available in lmstudio or omlx, waiting for that to streamline the experience a bit. But so far, the best i had till now.
I still haven't found a use case for Qwen3.8 27B that Qwen 3.6 35b A3b (MoE) is better at. I can get at most 40 tokens/second with 27B, but I get around 90 tokens/second with the MoE and it seems to be a more capable model.
I guess I should still keep experimenting though. Maybe I'm just not using a dense model correctly.
if you have 128 GB, you could use Qwen Flash Next at some reasonable quant, with the new SSD hack for only keeping some of the model resident in memory.
Does anyone actually pay API costs out of their own pocket? It's about 10x cheaper to just get a codex or chat gpt subscription, it's so heavily subsidized compared to the API that I'm sure it would be cheaper to use frontier models on a subscription plan rather than paying API prices for deepseek flash.
I do. For local dev work, I'm mostly using jetbrains' Junie, I can swap between a collection of models from google, openai, anthrophic.
I've had more than a few people tell me "oh, it's so much cheaper to use a $20 claude account" or "i've never hit a limit ever using my openai". Inevitably.. I end up reading/hearing "oh, I need to give it another couple hours to start using it again"... I've never hit that with my approach, even if it's costing me a bit more. Being able to work when I want when I have time has some value.
I also have openai and anthropic direct API billing set up for hosted and client projects that need to call out to an LLM service.
Why not use a codex or claude subscription? If you use the entire usage allotment on the $200 plan it's about $2,000 in equivalent API costs. Switching providers may be valuable but it's quite literally an order of magnitude cheaper.
That's... $200. That's a lot for me. And I'm tied in to one set of models. Using something else (like Junie) I can swap and try models from multiple vendors.
I would investigate if the $20/month plan is good enough for your use: The token-based plans usually work out to be much more expensive than the monthly ones. Or get both the Anthropic and Codex $20/month plans. I don't know if this makes sense compared to your current use case, but there is a very good chance it does given how much the per-token plans cost. The primary use case for them is enterprises that aren't allowed to use the monthly subscriptions.
Would you mind sharing how much it's costing you? I've been wanting to use APIs from within Intellij, but I hesitate because of uncertainty about the cost.
Junie is $30/month, and I get $35 in credits. There are times I go over the limit, and you can 'top up' in increments of $5 (min). On a busy month (for me) I may have spent and extra $40. Initially I didn't hit the limit. As it's more of a daily driver, I'm averaging an extra $20, so... maybe $50/month. Give it a shot - the CLI version made it more usable.
I thought Air was JB's multi-model interface? What is Junie? (I see the buttons, but am very confused by JB's AI offerings in general)
Is it worth the ~10x extra cost over the subscriptions? (This is obviously a leading question). Also, I think you can use OpenAI's subcription login with Air, but not Claude's.
Junie is their agent(?) that can connect to multiple providers. AFAIK you can bring-your-own-key or buy a subscription from them that lets you use your credits across multiple vendors (openai, anthropic, google) across a handful of models from each vendor. So yeah, you can use your openai credentials in junie vs paying for jetbrains junie credits.
Is it 'worth it'? I dunno. I've not used dedicated subscriptions. I adopted Junie when it first came out, and it's been 'good enough'. As I say, I've never hit the 'oh, i have to stop for several hours until they let me in again' wall because of just using the API approach.
As my monthly spend goes up, I might try something else out, but... gpt6-luna just came out. That is likely going to cut my spend already.
I do. Sharing training data with OpenAI gives me a lot of complementary tokens. I go above that but it's still quite economical and I pick the right model for the task (Luna for most).
Same thing. I'm sure a subscription would be better value if I had enough usage, but I don't. With Deepseek or similar a few $ is quite a bit of usage already.
I load OpenRouter up and use models like GLM Flash 5.3, DeepSeek Flash 4.1, Luna, etc. And I often have random niche needs where I need a handful of calls for say, a really good image reader like Gemini Flash 3.8 or whatever. You can do a lot with $25 on openrouter or direct to chinese providers. I am cautious about what data I send overseas, but also like... just because it is in China does not inherently mean it is any less secure than a US provider.
I can't remember the last time any real recourse has mattered for companies getting breached or mishandling my data. Their stock just goes up and the govt just shrugs.
I pay $100 for Codex and it last about a day in the weekly limit - mostly Astra and Sol.
Then i got $20 into DeepSeek and i've been using those $20 for two weeks every day now. Use case is automating computer/browser use - Astra is really good at it, but very expensive, Sol and Luna haven't been that great at it, Deepseek as at about 80% of Astra but lasts forever.
I'm using the Deepseek harness with the deepseek computer use and deepseek browser integration, it's somewhat janky but it gets the job done.
Also for coding i use OMP directly with Deepseek.
It's been rather awesome because even maxed out $20 gets you very far in between Codex resets.
I also tried developing a small game where I have Astra use as many Deepseek workers and it needs and it only does the 3D objects and high level architecture and integration, but it goes way too quickly to see good results yet.
I do, but via OpenRouter. Outside of work my use cases are small and cheaper models do great job at those. I noticed even if I "burn tokens like crazy" I still pay less than any subscription available (a few $ a month).
But I guess if I had an agent vibecoding on it's own, I'd go with subscription instantly.
You can't use an API key on subscriptions, but I've gotten around it using the `codex exec` command to run requests outside the CLI or GUI if you're already authenticated on that machine. Won't work for all cases, but I've never ran into a limitation in my use case of not having an API key.
I’ll look into this. But there is no universal solution. I find this to be quite limiting. I understand this is on purpose. If you want to automate, you need to pay more to have an api key.
It’s a good thing we have access to those very cheap and useful Chinese models.
Thanks! I have two mac minis: one with 8 gb and the other with 24 gb (left with 17 gb useable ram); I can't run any open model on them. So, ordered a Mac 5 pro with 128 GB. Can't wait for its arrival.
I'd really like something that's more oriented around subscription fees.
If I want to spend $100 on LLMs next month, what should I do? Get Claude because Opus 5.5/Fable 5.1 are scoring well? Get Grok because 4.7 is supposedly a good mix of competence and cost? Try out a Chinese model? Don't do a subscription at all like this site is saying?
What actual usage you want ? Because imho the answer is very different if you want code, or computer use, or vision, or text analysis, or ...
The subscription from frontier labs can handle all that (though each has its best strenght, they all perform well on most things), but it's usually a bit of a waste compared to taking the best one for you.
I've been running a quant/tune of Qwen3.8 27B on my M1 Max 32gb MacBook. That plus a good pi setup is having great results. I've used a full q8 of the model before and I dont see a real difference other than how slow it is. But leaving it running overnight on tasks is working great. It is currently debugging some issues in a native Mac Swift application and getting through the list of issues just fine.
I am running it on M3 pro. Works great except for prefill speed which makes it slow for many coding tasks. The newer generation of macs are promising but as a cost-sensitive user, I am also looking into cheaper 32 GB gpus from intel, AMD, nvidia. Eventually, I think these class of models will work well for most coding usecases especially given that I certainly want to have some control of the code generated.
1. In real life, most of us use token packages like OpenCode Go etc.
It would be handy to have a site like this one that takes into account the various deals and attempts to calculate the number of tokens per monthly fee for a chosen model. I realize this makes the task a lot more difficult.
2. It would be handy to have a chart like that for the AI hardware that people own. It helps you decide which model to run (resulting in different levels of intelligence and speed). Also difficult to please everyone (preprocessing vs token generation for example) and to keep updated!
I found https://llm-list.com/ yesterday and when I had a detailed look, I quickly found outdated entries, for example looking at GLM 5.3 flash it listed several providers as "free" that weren't free any longer.
As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.
For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.
No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.
Cost per Token is not representative of the actual cost incurred because a cheaper model can waste a lot of tokens to accomplish the same task. Also this does not consider cost for cached tokens which can often make a "cheaper" model more expensive even when total tokens spent are constant
It compares Artificial Analysis scores against usage limits on Opencode Go, so I can see where to waste my quota most efficiently. Updated whenever I feel like it.
More benchmark noise. They’re already problematic per se, and the error bars only get bigger by aggregating and adding costs on top of it.
Create your own, private evaluation system that reflects your use and your constraints. Focus on hard cases you encountered, find cases that break on one tier and not on another.
Exactly, according to these benchmarks Gemini 3.8 Flash is the best option at that budget level for coding, but in real world testing, I found DeepSeek flash to perform far better at a much lower price.
Is there a cheaper model than Gemini 3.8 Flash (High) that maybe/kind-of is on-par with it? For me it works really good but hit the limit in two hours tops... last week was the first time I hit the weekly limit and had to wait 4 days... Claude patches OK, but that is also getting drained really fast these days...
yeah I enjoy the speed of Gemini, but I also just have the low tier one (I use the 20x Claude and Codex subs for most of my work). For iteration, Gemini is so much fun, but Opus 5.5 is quite fast, as is sol. If you're on a budget, deepseek does look great
Yeah, I mean, I think in theory I could push it to the next tier on Gemini, but at the same time I wouldn't mind trying something else, since maybe my workloads are not really that smart and I am wasting a lot of computational power on something a cheaper model with similar capabilities can do.
If you maintain this over time, maybe include other sources than just AA and update the design to look less like zero-shot claude styling (I know that font! I know that color! Lol) it's genuinely useful :)
Coding and math graphs are very interesting. Extremely cheap models make it into the upper echelon, delivering 90% of the performance for 1% of the price compared to the #1.
I guess it’s hard to actually calculate, but most people are probably on subscription plans with Anthropic or OpenAI and there’s not any good resources to estimate which models are the most “usage limit” efficient. I suppose you can sort of correlate raw API costs with how much usage a model would probably drain, but it would be nice if there was a reliable place to get that info.
Fun, but this seems to assume all LLM run on SAAS subscriptions. I would like to compare this to local LLM costs by converting my usage load x hardware costs into a token price. In addition if it games the LLM so often, these eventually are optimized and basically cheat on the score.
What's impeding a lab from releasing its own new model, pricing it really low for the beautiful Pareto plot, accompanied by phrases VC love like "establishing a new frontier in cost", to just then raise prices back up?
They don't even have to "raise the prices back up". They just need to ensure that the number of output tokens goes up needlessly. Just make the model more talkative (via RL?) and cha-ching!
I find it interesting that with the given metric comparison, for coding at min 50 strength, every frontier model brand is from a distinct vendor: Ling, Qwen, Gemini, Muse, Grok, GPT, and Claude in increasing value.
all I ever wanted is an updated website where I can see the best models I can run on my different devices locally, I don't get why people are throwing money at these companies
Wouldn't be so sure, at least not for a GPU-based system. Quick math for a 5090 running at around 500W generating 100 tokens per second (reasonable for Qwen 3.8-27B) is around 2-3 kWh for a million tokens, which is around $0.60 for some mix of off/on peak electricity rates.
This is in the same ballpark for that same model on openrouter (https://openrouter.ai/qwen/qwen3.8-27b), highly dependent on input/output mix. Deepseek is a much more capable model that you can't run locally on normal hardware, and their rates are insanely cheap ($0.04 / $1 per 1M).
And so far I haven't considered the cost of the hardware. I happen to have a gaming PC that can be put to use on inference when not gaming, but given these numbers I don't think I would buy new hardware to do inference at home. Unless my math is wrong, it seems you're way better off paying for Deepseek than running Qwen or some other locally runnable model yourself. Of course if you have specific privacy requirements or prefer something unique about a particular model you can run locally, the equation changes.
Depends on what level of intelligence you're wanting to use. A vanishingly small number of people can or would want to go to the hardware expense of running something like GLM 5.3 Flash, much less something like K3.
And if you want Astra/Fable/Opus frontier level, then there's no option at all.
But if you don't need that, or you don't need speed... That opens up the discussion. I've been impressed even with how Siri's been doing with the Apple Foundation Models in MacOS/iOS 27 given how small they are.
Edit: I can't even fully spec the M5 Ultra Mac Studio you'd need for GLM5.3 Flash since 512GB isn't available yet, but it's already at $9500 for 256GB RAM.
> A vanishingly small number of people can or would want to go to the hardware expense of running something like GLM 5.3 Flash, much less something like K3.
It's probably worth letting the user specify their actual costs in such a tool. I run a Framework Desktop 128GB that I bought before memory prices got crazy; the current retail price is almost double what I actually paid a year ago.
I am sorry if this looks out of the blue, but I was trying to answer to an old thread, and HackerNews seems to lock old threads and doesn't allow for private messages (unless I am too dumb to figure it out, which is always a possibility).
Anyway, I wanted to reply to the very thoughtful comment you left my after I inquired about your responsibilities at a co/op(1) and I wanted to tell you that I am very proud of you and that you are the type of "hacker" that I aspire to be :)
I hope this doesn't come as too forward and I wish you anything but the best.
Best comparison that has occurred to me is the cost a loaf of bread's ingredients might be slightly cheaper than a baked loaf, depending on how you source it. At home you get total control and know what's going in to it. Yet bake at home is still a niche, perhaps a hobby. So I say as someone who's spent hundreds of hours tinkering with local inference, go for it for anyone reading. But most people just want ... some slices of bread, you know?
You're better off going directly to artificial analysis, this is a feature-poor/misleading/outdated repackaging
Ex. this type of price estimation is quite naive - some models can require 2-3x the number of tokens to achieve the same level of intelligence. Artificial Analysis' own cost per task is a more fair estimation of cost.
Agree :) direct link for those curious:
https://artificialanalysis.ai/models#price-cost
https://artificialanalysis.ai/models?models=gpt-6-luna%2Cglm...
It's kind of crazy that claude-opus-5.5 (max) uses 119k tokens per "intelligence index task" while gpt-6-astra (max) uses 27k.
https://artificialanalysis.ai/models?cost=intelligence-vs-co...
Exactly. The only metric you need to look at is the cost per task and take your pick from the models that match your budget there.
Spark is such a bad model, you are far better off using Astra Medium than using Spark.
Also cache price is crucial, especially for coding tasks (which, really, are where you really start to care the most about unbounded costs)
The analysis might be better on Artifical Analysis' however the way the information is presented by terrydjony is so much better. I'm able to get the exact actionable info I want. Whereas on artificial Analysis I'm analyzing and figuring it out.
It's the equivalent of ai slop writing in a data format.
If you have a 24-64GB mac, consider running Qwen3.8 27B locally at night. It's a bit slower to run locally, but if you're sleeping it's less of a problem.
Depending on your memory, you'll need to use the weaker Q4 versions but they still perform well.
It ranks higher than GPT-5.3 Codex (xhigh) or Claude Opus 4.6 (max) so is great for pairing with https://github.com/kunchenguid/gnhf for nightly experimentation, cleanup, or recommendation lists for in the morning.
I have all sorts of local compute, and local models fairly capable the Frontier models still way more capable/faster and local electricity consumption is something else. Good thing it is getting colder around here.
I often pair them up, and I have an Astra or Sol work as a supervisor and reviewer while Qwen 27B FP8 or Qwen 3.8 Flash Next implements things. I mostly do it as an experiment, just to see what kind of level of autonomy I can get, and they are slow to getting a decent reviewed outcome despite Qwen27B running at 100+ tps and 3.5-4K prefill rates and Qwen3.8 Next at 40 tps and 1-2K prefill. I've been also using similar approach more with OMP, not just the straight Pi harness. And OMP seems to be slower because it has more guardrails. OMP has an interesting feature where you can assign a better LLM as an advisor, wehere it just sort of monitors the progress and injects guidance. And it definitely helps, but one has to be careful. It actually turns out to be expensive if the cache reads are expensive. I learned it the hard way. Where on Fireworks' API, the cache rates for GLM 5.3 flash are quite a bit more expensive than for DeepSeek, and a simple runs ended up costing me three bucks in oversight. So a better way is to have a Frontier model running a separate tmux pane and just directing it to Wake up every 10 minutes, take a peek at what's going on, review the milestones give feedback and then sleep. This turns out to be pretty decent cost saving strategy when quote needs to be stretched. Paradoxically, OpenAI tightening up their quota allowance once they released Astra actually pushed me into all these sorts of experiments, and it's actually been interesting. I've been exploring all these smaller flash models, and it's been nice. I do like using local LLMs for chore type tasks that are just mostly information gathering, post-session reviews, stuff like that.
There also exists a $600-800 GPU that can run Qwen 27B 3.8 @ like 60 t/s for around 200W of energy.
I find Qwen Flash Next quite competent as well. 27B is a solid worker like you said. If you batch work and let them crank they do remarkably well. I am working on.
I started using Herdr and taught my agents to use it. So I use OMP loop and or goal, and it has a review cycles to wake up an Opus or Sol reviewer to make sure nothing is going off the rails with a local qwen flash next coordinating for me. I kinda prefer Sol, it seems like a more patient and thorough model, especially Sol 6, but Opus 5.5 is really good and its voice and attitude is not as grating as Opus 5 for sure.
I have a few V100 GPU running Qwen 27B and they do all the work overnight. Not quite the same speeds you have yet, but this is V100 machine and an old gaming machine with 16GB 4080 and a handful of 32GB V100s... all in less than 3K (ignoring that my gaming machine is 3 years old, but runs qwen flash next for free now as I game not a lot) for my little "we have AI at home" projects and there is a lot of interest in these old GPU now because they are rolling out of data centers now.
For my local work and personal projects... they just seem to be getting done in this setup. Every few days I sit down and do a big cycle with astra/fable/opus batch things up. I have projects that are basically "i want to see what happens" to "I want this to be good, I understand the code". Some of the throwaway projects that have just kind of magically finished more or less how I wanted have been great.
What do you mean by "local electricity consumption is something else"? Doesn't an M3 Pro for example draw about as much power as a bright incandescent lightbulb for a maxed out gpu workload (~100W)? That's less than a tenth of what a frontier model will use in the datacenter (which I believe are racks of BlackWell or Vera Lynn GPUs, each using 500W+).
Because M3 Pro is not a real thing as far as actual agentic workflows go. I run dual R9700 boxes those idle at 150W Because of a Ryzen AM5, and I run Spark boxes, which are decent But still idle at 45-50 watts a pop. My favorite, to be honest, is a 5090 with a Qwen 27B because that one is good for quick hitters and flies, But again, the box itself idles at 140 watts. So it's really the idle power that I don't like And it's too much of an inconvenience to power boxes down and power them back on, so they just end up running and sucking electricity. I am working on getting some sort of a smarter proxy setup where I would give boxes time to wake up and go to the cloud while they're waking up. So the whole point of having these things running in the background is that you do want them to be almost always available. So power consumption is definitely an issue.
On my 64gb m3 max qwen3.8 27b has been great for planning and then letting qwen3.6 35ba3b actually implement the planned changes.
You might be interested in https://prismml.com/news/bonsai-2-27b
This looked exciting until I read that it gets stuck in loops and generally wasn't a useful model
Please cite a reputable source for this claim, and keep in mind we are talking about Bonsai 2 and not the original Bonsai.
Even on the original one, the authors have replied to such issues with working solutions: https://github.com/PrismML-Eng/Bonsai-demo/issues/131
for 32gb, this is my model of reference now, you have to run it with a patched version of llama and is still not available in lmstudio or omlx, waiting for that to streamline the experience a bit. But so far, the best i had till now.
It looks like upstream work is underway: https://github.com/ggml-org/llama.cpp/issues/29058
The sheer number of geniuses working so fast on this repo never ceases to amaze me.
If you're on MacOS, with atleast an M3 chip and 32GB, you should look at the splash engine.
GNHF seems exactly what I've been aiming for to handle overnight tasks.
It's _so_ good, I no longer bother with Sonnet and use it locally for everything.
Consider bumping reasoning down to Medium as a default though, I agree with simonw it over thinks https://simonwillison.net/2026/Aug/16/qwen-38-27b/
Isn't there a way to set up a thinking budget so it automatically tells the model "Conclude your reasoning now and provide the final answer." ?
I still haven't found a use case for Qwen3.8 27B that Qwen 3.6 35b A3b (MoE) is better at. I can get at most 40 tokens/second with 27B, but I get around 90 tokens/second with the MoE and it seems to be a more capable model.
I guess I should still keep experimenting though. Maybe I'm just not using a dense model correctly.
What would you personally recommend for those that have 128 GB?
I have a Ryzen AI Max+ 395 with 128 GB for running those sort of "background task", and my sweet spot is currently Qwen3.8 Flash-Next IQ4 at 96 GB.
if you have 128 GB, you could use Qwen Flash Next at some reasonable quant, with the new SSD hack for only keeping some of the model resident in memory.
Does anyone actually pay API costs out of their own pocket? It's about 10x cheaper to just get a codex or chat gpt subscription, it's so heavily subsidized compared to the API that I'm sure it would be cheaper to use frontier models on a subscription plan rather than paying API prices for deepseek flash.
I do. For local dev work, I'm mostly using jetbrains' Junie, I can swap between a collection of models from google, openai, anthrophic.
I've had more than a few people tell me "oh, it's so much cheaper to use a $20 claude account" or "i've never hit a limit ever using my openai". Inevitably.. I end up reading/hearing "oh, I need to give it another couple hours to start using it again"... I've never hit that with my approach, even if it's costing me a bit more. Being able to work when I want when I have time has some value.
I also have openai and anthropic direct API billing set up for hosted and client projects that need to call out to an LLM service.
Why not use a codex or claude subscription? If you use the entire usage allotment on the $200 plan it's about $2,000 in equivalent API costs. Switching providers may be valuable but it's quite literally an order of magnitude cheaper.
That's... $200. That's a lot for me. And I'm tied in to one set of models. Using something else (like Junie) I can swap and try models from multiple vendors.
I would investigate if the $20/month plan is good enough for your use: The token-based plans usually work out to be much more expensive than the monthly ones. Or get both the Anthropic and Codex $20/month plans. I don't know if this makes sense compared to your current use case, but there is a very good chance it does given how much the per-token plans cost. The primary use case for them is enterprises that aren't allowed to use the monthly subscriptions.
I only used that number because that's what I use. If you get the $20 codex plan that's equivalent to $100 in API costs, still a much better deal.
Is it equivalent if I'm time bound and have to stop working for X hours?
>it's costing me a bit more
Would you mind sharing how much it's costing you? I've been wanting to use APIs from within Intellij, but I hesitate because of uncertainty about the cost.
Junie is $30/month, and I get $35 in credits. There are times I go over the limit, and you can 'top up' in increments of $5 (min). On a busy month (for me) I may have spent and extra $40. Initially I didn't hit the limit. As it's more of a daily driver, I'm averaging an extra $20, so... maybe $50/month. Give it a shot - the CLI version made it more usable.
I thought Air was JB's multi-model interface? What is Junie? (I see the buttons, but am very confused by JB's AI offerings in general)
Is it worth the ~10x extra cost over the subscriptions? (This is obviously a leading question). Also, I think you can use OpenAI's subcription login with Air, but not Claude's.
It IS confusing. I'm not even using Air.
Junie is their agent(?) that can connect to multiple providers. AFAIK you can bring-your-own-key or buy a subscription from them that lets you use your credits across multiple vendors (openai, anthropic, google) across a handful of models from each vendor. So yeah, you can use your openai credentials in junie vs paying for jetbrains junie credits.
Is it 'worth it'? I dunno. I've not used dedicated subscriptions. I adopted Junie when it first came out, and it's been 'good enough'. As I say, I've never hit the 'oh, i have to stop for several hours until they let me in again' wall because of just using the API approach.
As my monthly spend goes up, I might try something else out, but... gpt6-luna just came out. That is likely going to cut my spend already.
I do. Sharing training data with OpenAI gives me a lot of complementary tokens. I go above that but it's still quite economical and I pick the right model for the task (Luna for most).
20$ subscriptions offer “top up” too I’m pretty sure.
I do, 3-4$ a month of deep seek is enough for my usage
Same thing. I'm sure a subscription would be better value if I had enough usage, but I don't. With Deepseek or similar a few $ is quite a bit of usage already.
I load OpenRouter up and use models like GLM Flash 5.3, DeepSeek Flash 4.1, Luna, etc. And I often have random niche needs where I need a handful of calls for say, a really good image reader like Gemini Flash 3.8 or whatever. You can do a lot with $25 on openrouter or direct to chinese providers. I am cautious about what data I send overseas, but also like... just because it is in China does not inherently mean it is any less secure than a US provider.
I can't remember the last time any real recourse has mattered for companies getting breached or mishandling my data. Their stock just goes up and the govt just shrugs.
I pay $100 for Codex and it last about a day in the weekly limit - mostly Astra and Sol.
Then i got $20 into DeepSeek and i've been using those $20 for two weeks every day now. Use case is automating computer/browser use - Astra is really good at it, but very expensive, Sol and Luna haven't been that great at it, Deepseek as at about 80% of Astra but lasts forever.
Curious to hear more details about your harness and setup if you’re open to sharing.
I'm using the Deepseek harness with the deepseek computer use and deepseek browser integration, it's somewhat janky but it gets the job done.
Also for coding i use OMP directly with Deepseek.
It's been rather awesome because even maxed out $20 gets you very far in between Codex resets.
I also tried developing a small game where I have Astra use as many Deepseek workers and it needs and it only does the 3D objects and high level architecture and integration, but it goes way too quickly to see good results yet.
[flagged]
I do, but via OpenRouter. Outside of work my use cases are small and cheaper models do great job at those. I noticed even if I "burn tokens like crazy" I still pay less than any subscription available (a few $ a month).
But I guess if I had an agent vibecoding on it's own, I'd go with subscription instantly.
I use local LLMs on my Mac Mini.
Otherwise DeepSeek Flash 4.1 is dirt cheap (other "Flash" models are not that expensive either). I pay (very few dollars) out of my own pocket.
There are many things where having an API Key is necessary.
Maybe I’ve missed the boat though: is there now a method to use an api key to access a subscription?
You can't use an API key on subscriptions, but I've gotten around it using the `codex exec` command to run requests outside the CLI or GUI if you're already authenticated on that machine. Won't work for all cases, but I've never ran into a limitation in my use case of not having an API key.
I’ll look into this. But there is no universal solution. I find this to be quite limiting. I understand this is on purpose. If you want to automate, you need to pay more to have an api key.
It’s a good thing we have access to those very cheap and useful Chinese models.
> I use local LLMs on my Mac Mini.
which ones do you use?
I use the following models:
I have 48G.The MacMini is used solely for inference. llama.cpp + llama-swap.
People are saying local models are crap and serves no purposes.
I use them to help me spell check, write emails, write text messages, write JIRA tickets, write PR comments, etc.
I also use them in coding agents to complete different tasks.
I find them quite useful!
Thanks! I have two mac minis: one with 8 gb and the other with 24 gb (left with 17 gb useable ram); I can't run any open model on them. So, ordered a Mac 5 pro with 128 GB. Can't wait for its arrival.
128G of RAM? I envy you.
You will be able to run some pretty impressive models locally. Good for you.
Large enterprises pay per token even through a ChatGPT “membership”, for example.
Which is why I said "out of their own pocket"
>heavily subsidized
Do you think this will be a problem in the future?
I'd really like something that's more oriented around subscription fees.
If I want to spend $100 on LLMs next month, what should I do? Get Claude because Opus 5.5/Fable 5.1 are scoring well? Get Grok because 4.7 is supposedly a good mix of competence and cost? Try out a Chinese model? Don't do a subscription at all like this site is saying?
Or get a 20$ subscription from each major provider and have 40$ spare for openrouter credits. That’s what I’m doing.
I like this idea. Is it easy to pass work around between the models?
What actual usage you want ? Because imho the answer is very different if you want code, or computer use, or vision, or text analysis, or ...
The subscription from frontier labs can handle all that (though each has its best strenght, they all perform well on most things), but it's usually a bit of a waste compared to taking the best one for you.
The problem is that subscriptions with AI models get a bit 'vague' on what you get and how (or when) you might be rate-limited.
I've been running a quant/tune of Qwen3.8 27B on my M1 Max 32gb MacBook. That plus a good pi setup is having great results. I've used a full q8 of the model before and I dont see a real difference other than how slow it is. But leaving it running overnight on tasks is working great. It is currently debugging some issues in a native Mac Swift application and getting through the list of issues just fine.
This is the one that works good for me on 32gb:
https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
Specifically this one: Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
I am running it on M3 pro. Works great except for prefill speed which makes it slow for many coding tasks. The newer generation of macs are promising but as a cost-sensitive user, I am also looking into cheaper 32 GB gpus from intel, AMD, nvidia. Eventually, I think these class of models will work well for most coding usecases especially given that I certainly want to have some control of the code generated.
1. In real life, most of us use token packages like OpenCode Go etc.
It would be handy to have a site like this one that takes into account the various deals and attempts to calculate the number of tokens per monthly fee for a chosen model. I realize this makes the task a lot more difficult.
2. It would be handy to have a chart like that for the AI hardware that people own. It helps you decide which model to run (resulting in different levels of intelligence and speed). Also difficult to please everyone (preprocessing vs token generation for example) and to keep updated!
I found https://llm-list.com/ yesterday and when I had a detailed look, I quickly found outdated entries, for example looking at GLM 5.3 flash it listed several providers as "free" that weren't free any longer.
As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.
For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.
No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.
Cost per token is an extremely naive way to index cost, renders this chart essentially meaningless.
Cost per Token is not representative of the actual cost incurred because a cheaper model can waste a lot of tokens to accomplish the same task. Also this does not consider cost for cached tokens which can often make a "cheaper" model more expensive even when total tokens spent are constant
This definition of cost is not particularly useful. You want cost per task, not cost per 1m tokens. Artificial Analysis does a good job of this.
I wrote (i.e. let AI write) something like this for Opencode Go: https://6bj.de/aaogo/go_value_report.html
It compares Artificial Analysis scores against usage limits on Opencode Go, so I can see where to waste my quota most efficiently. Updated whenever I feel like it.
More benchmark noise. They’re already problematic per se, and the error bars only get bigger by aggregating and adding costs on top of it.
Create your own, private evaluation system that reflects your use and your constraints. Focus on hard cases you encountered, find cases that break on one tier and not on another.
Exactly, according to these benchmarks Gemini 3.8 Flash is the best option at that budget level for coding, but in real world testing, I found DeepSeek flash to perform far better at a much lower price.
I don’t think they have 4.1 flash in the graph, fwiw
Is there a cheaper model than Gemini 3.8 Flash (High) that maybe/kind-of is on-par with it? For me it works really good but hit the limit in two hours tops... last week was the first time I hit the weekly limit and had to wait 4 days... Claude patches OK, but that is also getting drained really fast these days...
DeepSeek 4.1 Flash is your answer.
Ah OK, yeah, I was actually going to go for that earlier, I'll check it out once I get home, thanks!
is this on the $20-ish sub? I hear ultra lasts for a very long time
Yeah, 20... I was going to try Deepseek later on, just chuck 20 in there and see how good it does, and how far I can go as well...
yeah I enjoy the speed of Gemini, but I also just have the low tier one (I use the 20x Claude and Codex subs for most of my work). For iteration, Gemini is so much fun, but Opus 5.5 is quite fast, as is sol. If you're on a budget, deepseek does look great
Yeah, I mean, I think in theory I could push it to the next tier on Gemini, but at the same time I wouldn't mind trying something else, since maybe my workloads are not really that smart and I am wasting a lot of computational power on something a cheaper model with similar capabilities can do.
If you maintain this over time, maybe include other sources than just AA and update the design to look less like zero-shot claude styling (I know that font! I know that color! Lol) it's genuinely useful :)
Updated daily and yet the frontier for math is listed as gpt-5.2
Coding and math graphs are very interesting. Extremely cheap models make it into the upper echelon, delivering 90% of the performance for 1% of the price compared to the #1.
The coding and math tabs seem to be missing the latest models...
Math in particular is quite far behind. GPT 5.2 is recommended as the best for highest cost. Really?
I guess it’s hard to actually calculate, but most people are probably on subscription plans with Anthropic or OpenAI and there’s not any good resources to estimate which models are the most “usage limit” efficient. I suppose you can sort of correlate raw API costs with how much usage a model would probably drain, but it would be nice if there was a reliable place to get that info.
Some of the charts are not uptodate.
For example "Raw capability: Math Index", shows GPT-5.2 as highest. It does not even mention astra, of fable.
I tried to build a very similar “frontier” [1] for myself and also include writing (which is highly debatable)
[1] https://philippdubach.com/posts/jev-model-router-for-pi/
Fun, but this seems to assume all LLM run on SAAS subscriptions. I would like to compare this to local LLM costs by converting my usage load x hardware costs into a token price. In addition if it games the LLM so often, these eventually are optimized and basically cheat on the score.
It's missing Opus 5.5 which was released over a day ago (and also is clearly on the pareto frontier).
Don't know if they updated it since your comment, but for me it's on the graph – and on the Pareto frontier indeed.
It was expert timing on the commenter's and the developer's part. Opus 5.5 still doesn't show under coding though.
Opus 5.5 is indeed on the Intelligence tab, but I don't see it on the Coding or Math tabs.
For intelligence only as of 8:51 MT
What's impeding a lab from releasing its own new model, pricing it really low for the beautiful Pareto plot, accompanied by phrases VC love like "establishing a new frontier in cost", to just then raise prices back up?
They don't even have to "raise the prices back up". They just need to ensure that the number of output tokens goes up needlessly. Just make the model more talkative (via RL?) and cha-ching!
Oh yeah totally, good point. I remember there was controversy around Claude tokenizer generating more tokens lol
I won’t touch Meta models. I simply don’t trust or like Zuck and especially Wang
I find it interesting that with the given metric comparison, for coding at min 50 strength, every frontier model brand is from a distinct vendor: Ling, Qwen, Gemini, Muse, Grok, GPT, and Claude in increasing value.
> "$0.23 to under $0.24 - Qwen3.8-Flash-Next"
There are only 6 categories in the site, but a category entirely created out of thin air for Qwen3.8-Flash-Next?
Literally "$0.23 to under $0.24" means EXACTLY $0.23.
all I ever wanted is an updated website where I can see the best models I can run on my different devices locally, I don't get why people are throwing money at these companies
Guessing that most people don't have machines powerful enough to run good-enough models locally.
Interested to see the Coding and Math scores for Grok 4.7 when they’re available. Grok 4.6 is apparently already on the frontier for Coding.
I would really like something similar for local ai where the x axis is vram.
The page looks Claude generated and ranks Opus at the top.
I'm sure that local LLM will be far cheaper
Wouldn't be so sure, at least not for a GPU-based system. Quick math for a 5090 running at around 500W generating 100 tokens per second (reasonable for Qwen 3.8-27B) is around 2-3 kWh for a million tokens, which is around $0.60 for some mix of off/on peak electricity rates.
This is in the same ballpark for that same model on openrouter (https://openrouter.ai/qwen/qwen3.8-27b), highly dependent on input/output mix. Deepseek is a much more capable model that you can't run locally on normal hardware, and their rates are insanely cheap ($0.04 / $1 per 1M).
And so far I haven't considered the cost of the hardware. I happen to have a gaming PC that can be put to use on inference when not gaming, but given these numbers I don't think I would buy new hardware to do inference at home. Unless my math is wrong, it seems you're way better off paying for Deepseek than running Qwen or some other locally runnable model yourself. Of course if you have specific privacy requirements or prefer something unique about a particular model you can run locally, the equation changes.
Depends on what level of intelligence you're wanting to use. A vanishingly small number of people can or would want to go to the hardware expense of running something like GLM 5.3 Flash, much less something like K3.
And if you want Astra/Fable/Opus frontier level, then there's no option at all.
But if you don't need that, or you don't need speed... That opens up the discussion. I've been impressed even with how Siri's been doing with the Apple Foundation Models in MacOS/iOS 27 given how small they are.
Edit: I can't even fully spec the M5 Ultra Mac Studio you'd need for GLM5.3 Flash since 512GB isn't available yet, but it's already at $9500 for 256GB RAM.
> A vanishingly small number of people can or would want to go to the hardware expense of running something like GLM 5.3 Flash, much less something like K3.
It's probably worth letting the user specify their actual costs in such a tool. I run a Framework Desktop 128GB that I bought before memory prices got crazy; the current retail price is almost double what I actually paid a year ago.
Dear person,
Anyway, I wanted to reply to the very thoughtful comment you left my after I inquired about your responsibilities at a co/op(1) and I wanted to tell you that I am very proud of you and that you are the type of "hacker" that I aspire to be :)I hope this doesn't come as too forward and I wish you anything but the best.
(1) : https://news.ycombinator.com/item?id=48383220
Best comparison that has occurred to me is the cost a loaf of bread's ingredients might be slightly cheaper than a baked loaf, depending on how you source it. At home you get total control and know what's going in to it. Yet bake at home is still a niche, perhaps a hobby. So I say as someone who's spent hundreds of hours tinkering with local inference, go for it for anyone reading. But most people just want ... some slices of bread, you know?
Astra worse than 5.6 Sol is crazy work
I’d love this but for every hardware
When a clanker is slopping up a webpage, why does it have to be #f4f3ee? Curious.
Luna at the bottom of intelligence is laughable...
[flagged]
[dead]