What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
As a counterpoint: in a complex project, Fable's "curiosity" may be exactly what you want for an exploration and planning stage - not just for the orchestrator that turns your prompt into different angles with which to explore, but for each subagent whose task is to search the codebase for one of those "angles." If you truly want no stone unturned, letting those subagents spawn their own discoveries, and recursively grow the surface area of the inquiry, then it's quite reasonable to want Fable throughout.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
It's tuned for the kinds of tasks where "just try" doesn't get good results.
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
Sub agents each have to read part of your code base again to get enough context for the task. And if they take too long, your orchestrator's context is no longer in cache so you pay full price for that again once the subagents finish
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
Yes but one of the key things about subagents is they keep all of their tool calls and exploration out of the parent context.
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
It’s funny too because I’ll ask fairly simple things and it’s fine, similarly simple question might spin up a bunch of sub agents and I don’t know why….
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
I'm pretty sure the system instructions are a function of your environment and not the same universally. That said, there should be a finite number of branches so still cacheable.
For a while everyone was saying sub agents is how you save tokens, use lower quality models with limited context to do simple parts of the job after a smart planning agent has put it all in place. Is that no longer true or is this just the result of sub agent being used at the wrong time?
No, you can definitely configure low cost search and apply subagents. CC and Codex do not. Not sure if this is to improve the reliability of their subagents, or just a play to increase user consumption.
> I still have PTSD from how ungodly terrible it was
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
This has been my experience as well. Something happened 2-3 months ago with Claude Code. It got slower, starting spinning and getting stuck more and more. I gave codex another shot out of my Claude frustrations, and have never looked back again.
Just tried Claude Code yesterday, and nope, it's the same old bad.
I think there's some setting to restrict the number of them, or maybe turn them off. Doesn't happen for me ~ever and it's not my $$ (work) so I haven't really looked at it much.
I find it useful for code reviews (spawn a subagent with minimal/no context to review X commit). Of course, this is more or less a shortcut that could be done with a seperate agent. Another use is multiple reviews at once if tokens are not an issue, with seperate "personas" or focuses. As far as implementation goes I have not seen any major usecase.
> What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
for subagents to be cheap/effective, you have to specify the size of those subagents; i.e. right now by default 5.6-sol spawns many 5.6-sol subagents. 5.4-mini as subagent saves me tons of tokens. 5.6-sol audits the work before accepting it, so there's not really a quality issue.
Subagents with a fat tailed latency distribution completely masks the trough filling that puts the most downwards pressure on per-token COGS.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
It's in the best interest for AI companies to gobble up tokens. I feel like every new release - Fable, etc - is just a way to extract more tokens/money.
My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions. This is supported by the fact that they won't let you use your sub on a different coding agent. I use pi btw.
Once I realized that Anthropic is a token merchant, I start to understand Anthropic’s decision more. They are always finding reasons for you to use more tokens through them unless the users revolt or demand some guardrails.
I've done a couple side by sides on web chat with the same prompt on Opus 4.6, 4.7, and 4.8 and the output gets longer/more verbose on version increment. The enerr variants are definitely much wordier.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
Seems unlikely they'd be this dumb. The way to get us to use more tokens is to make those tokens more useful, not less. Anthropic is full of people (including higher-ups) who know this.
But they gave us double the tokens! Then a limited time more usage! Then even more tokens "off peak" times! Then some new model released but apparently it inherently used 1.69x tokens! Then Fable is here but "it uses much more usage". But only until ~~the US banned it~~ ~~7th July~~ ~~19th July~~ who even knows.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
now reealize that LLMs are trained to produce tokens and like the halting problem, cant be trained not to produce tokens and youll realiE the AI labs are the perfect essential capitalist and like cancer, will keep growing useless tokens until it kills its host.
no amount of alignment will stop aomeone drom just shutting up.
The vast majority of my company's enterprise plan use is through Claude Code even though we have access to the API and could be using OpenCode instead.
I don't fully agree with the premise that they intentionally increase system prompts, but the enterprise plan usage is going to make that a huge income for Anthropic.
You're making the opposite argument. Anthropic is incentivized to use less tokens in Claude Code because people are paying a fixed monthly fee for subscriptions.
Generally, companies with >150 people can’t use subs. So yeah, it’s mostly a funnel for devs/small companies to eventually vet for the product and convince their enterprise to use it as well.
Flip side is customer psychology. Choosing a more expensive tier leaves better emotion.
Also i doubt there was jira ticket with “make llm more verbose”, rather ticket with “bug makes llms too verbose” gets prioritised taking revenue impact into account.
Pi is one of the ways out of this problem (OpenCode another) so I took it as an intentional reference as it is highly relevant. I also use Pi as my daily driver and I think it's a wise choice to figure out how to decouple yourself from lab-specific harnesses that you have little control or observability over.
the amount of system prompt wastage going on in orgs is insane. we identified 400k in annual burn for zero value in just one section of our large company.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
> This is supported by the fact that they won't let you use your sub on a different coding agent
I mean, that's a very weak argument? Isn't a much more plausible explanation that with your tooling you'll have more of a lock-in than with just your model?
After reading PUSH_AX's valid comment:
```
This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000
Are we measuring and caring about the right thing?
```
We will update the post to include:
1) A more in-depth task.
2) Qualitative results comparison.
3) As soon as possible, a reproduction of the inputs and outputs.
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls:
I often find myself annoyed when Opus fixes a typo in a comment and decides to run tests, lints and whenever else it can find to run. Often it will start by stashing current changes just to preemptively check if all tests were passing before.
And I can blame myself a bit because my rules do say: verify all changes with tests. But as there is that I in AI that is hyped which you’d think means it knows not to put tomatoes into fruit salad …
> [..] my rules do say: verify all changes with tests
I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint.
If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
Following rules like "verify all changes with tests" down to a tee is usually a desirable trait in LLMs. Personally I'd leave that behavior there (just like with humans for some tasks like aviation you have them go through checklists even if some stuff you can infer is not needed). But otherwise just make it "always run tests unless you're absolutely sure they can be skipped".
> prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
Recently switched to Codex after 6m in Claude. Codex seems more open, it’s easier to follow what the model is doing and the approvals have a better UX. Overall, it just feels more transparent. Cost of switching was close to 0.
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I use both now and agree they're basically interchangeable.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
CC went from sane defaults in late 2025 to feature scope creep early 2026. So more features might be good, but sounds like an ick for me. But I have zero prestige, I might switch back.
Maybe related to this minimalism, Pi doesn't come with most of the tools an LLM needs to function efficiently or effectively. I get that a blank slate is the paradigm, and you can add whatever you want, but it's too blank IMO.
I have a functional Pi config, mostly self-made (it has everything I want, incl. subagents, web search, a /btw command, and other misc. addons), and my system prompt is ~3k.
Sure, but you have to add almost everything, no? It deliberately only comes with read, write, edit, and bash. My point wasn't that you can't add stuff, but that I'd just rather use an harness that's a bit more full featured from the start.
(Pi is a bit like old 3D printing where fettling the printer to work is a central part of the hobby. I'd rather just buy a Prusa.)
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
Exactly! I just vibe coded (with GPT Sol and Claude whatever-number) my own agent, it's trivial to add now any feature I want - simply ask more powerful model to do it for you. I am happy with end result, however it looks indeed these tools are trained to increase token count - they do quite stupid token-spending steps while making code, but the code itself is also a bit weird - it's like they intentionally do code which is hard to modify on your own without using exactly those authoring models. Interestingly, when I am using DeepSeek with OpenCode, I don't see that - it understands my intent well enough and overall code quality is not bad. I recently switched to local Gemma 4, and I often switch (in opencode) to just that less powerful model, because it understands my intent and has enough skills to provide good quality solution although it's rather for small size projects, and for not coding from scratch, but it's also free and private. It feels slower than any big cloud model, so my model switching is probably most quickest path to robust end result :)
I wrote my own harness in Emacs and it’s completely ridiculous how well it works. Auto-compact is the only missing feature on my list. Claude‘s approach, if I understand it correctly, invalidates a lot of cached context, and I‘m thinking about a more cache-friendly strategy.
At a minimum, you need an inference endpoint: either cloud or local.
If going local, llama.cpp is going to be the more beginner friendly local inference engine that supports more processor types (AMD GPUs, Intel GPUs, CPUs, anything that supports Vulkan, not just Nvidia). LM Studio is a nice wrapper for this if you'd rather avoid cloning repo and compiling yourself, provided you don't mind closed source software; it's much less enshittified than Ollama.
If going local, you will also need model weights in the right format for your inference engine, and with a model that can fit on your hardware. This is going to be .GGUF files if you're using llama.cpp or a wrapper for it like LM Studio.
From there, pick a language, go look up the OpenAI /chat/completions API format (or Anthropic's "Responses" API format), create a DS or array or slice to store messages, and build a loop that accepts user input, formats it according to the API format, sends it to the inference server, retrieves and parses the response, adds the response to the DS/array/slice, and repeat.
There's a lot more beyond this - tool calling, other API formats (optionally), MCP servers, transport layers besides terminal stdin/stdout, permission models, starting with a system message, clearing your message stack correctly (hint: don't reset it mid tool-call), message compaction, web searching and page fetching, semantic search RAG over embeddings, memory layers - way too much to cover exhaustively in a single message.
I am forced to use cloude code at work but a good solution is to just use --system-prompt "" and be done with it. I wish they allowed for other harnesses.
Does it have any negative impact? If not, I’m not sure why this wouldn’t be the default behavior. It feels like Anthropic is just putting their foot on the scale to drive up costs or for the enterprise, or push consumers to higher subscription tiers.
A harness is a part of the intelligence stack. It's no longer about raw access to the model
Also, I have seriously used most harnesses - One feels like it's being built in a place that truly understands AI and where agentic engineering is headed. You might not like it, but peak performance exists in CC when it comes to orchestration of bulk parallel work / subagents. The open source agents are catching up or accell in different way (Im preferable to pi.dev), but I'm not sure they're architecting orchestration the right why.
Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow.
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.
Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc.
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
Claude Code is not just a harness. It is a different product. You pick the smallest subscription that allows you to do your work. My “multiplier” on a $100 subscription is 5+.
If you’re using API, on the other hand, there is absolutely no reason to use Claude Code, or Codex.
This is all heading in the right direction. Much of AI coding feels magical. But when the costs begin to accrue we start asking questions. We dig into it and try to understand what's going on. I can't help but feel Anthropic is "token maxing" from its side: it controls the levers and with every version upgrade it can build in its own token growth almost unbeknownst to the user. This actually harms it on the long run because it necessitates a cheaper option.
Claude Code sending 33k tokens before reading the prompt is the AI equivalent of a consultant who bills you for the time spent reading your email before they even open it.
The reasoning built into the models matter so much too. I recently swapped my Qwen3.6 27B to ThinkingLabs’ fine tune and it does what it publishes. I cut my token usage in half, which is a big deal since I only get ~20 TPS for token generation.
With Fable being per token instead of on the subs (unless they changed it again?), I decided to test Claude code on OpenRouter where I had some credits, with Opus 4.8 and Fable 5.
I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
As for context size and harnesses I did make a trivial bash agent based on this "agent in 50 lines" tutorial[0] recently, and found that for trivial work, it was about an order of magnitude cheaper and faster.
I haven't tested it on anything bigger but it doesn't seem to do the kind of proactive testing, that they do in bigger harnesses.
Codex at least has a system prompt that tells it not to consider a feature a complete until it has verified it. I'm not sure about Claude Code.
I suppose I could add that one line to the prompt, and it would get me much closer to agi :) I think Fable does this proactively even without a prompt, but I haven't tested that yet.
If Fable in my own harness is significantly cheaper than Claude Code, that would be very appealing. (I could actually afford to use it for most things!) But I think most of the cost comes from the testing it does. So we'll have to see.
Fable's subscription inclusion theoretically ends EOD today. Anthropic put a wishy-washy "if we have capacity we'll continue it" thing, and given how competitive GPT 5.6 Sol is, and it is included in OpenAI's subscription, I fully expect Anthropic to extend Fable or they will have a serious exodus on their hands.
Anthropic have extended Fable access again to July 19. The notice should pop up in your Claude Code now when you start a new session (also announced on the ClaudeDevs X account first).
Ah, thanks. It's been hard to plan around these last-minute changes. I rushed to implementation on a spec I should have spent more time on because of the looming deadline.
I've been trying various harnesses like Pi, OpenCode, Qwen Code, and Nanocoder. A common problem I keep running into is failed tool calls, regardless of the model. What is the best harness and on-device model combination right now?
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.
Pi.dev requires some plugins to work well. Using Qwen3.6-27B/35B locally at Q8, I was quite frustrated with failed tool calls and tried many things.
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
I feel like this article isn't saying much. Even with tools disabled, Claude Code still has a crap load of commands and other things that Claude (the model) should know the availability of since it's optimized for them. All of that has to be disabled if this is to be a real harness comparison. And of course the system prompt can be completely replaced, making it a no-brainer to use a more minimal prompt similar to OpenCode. And beyond that nothing else really matters because the rest (cache behavior, etc) lies with the provider's platform, not the harness.
Grok 4.5 is really fast, has more usage at $10/month than $20/month Claude pro, and Opus-level. Claude pro feels like a demo.
Claude is much better in OpenCode then in Claude Code, OpenCode is just better than Claude Code. Claude Code feels like a complete mess to use comparatively.
The Sleev (the project has been renamed to make a startup) creator was shilling their project in the OpenCode Discord. That person is very convinced they have something that no one has ever built before. They focused on token reduction without any real evals for capability impacts.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
It's definitely not unprecedented, but the plugin version is useful. Sleev seems like a nothingburger, I'm happy with the results I get from DCP already.
No surprise, I've noticed that "agents", not only CC (I am using Copilot) are trying to be "clever", searching for a lot of data. This is good for LLM providers as this eats a lot of tokens.
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
Given they're incentivized to increase token use, what guarantees that higher token use improves the effectiveness of the agent and isn't just artificial padding?
Well, nothing really. But I assume there can be some benefits to modifying context. For example, updating file contents or marking them as modified, summarization, injecting additional information, removing irrelevant tool call results, etc.
> Claude Code 2.1.207 and OpenCode 1.17.18, both pinned to claude-sonnet-4-5
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.
What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
As a counterpoint: in a complex project, Fable's "curiosity" may be exactly what you want for an exploration and planning stage - not just for the orchestrator that turns your prompt into different angles with which to explore, but for each subagent whose task is to search the codebase for one of those "angles." If you truly want no stone unturned, letting those subagents spawn their own discoveries, and recursively grow the surface area of the inquiry, then it's quite reasonable to want Fable throughout.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
It's tuned for the kinds of tasks where "just try" doesn't get good results.
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
Sub agents each have to read part of your code base again to get enough context for the task. And if they take too long, your orchestrator's context is no longer in cache so you pay full price for that again once the subagents finish
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
Yes but one of the key things about subagents is they keep all of their tool calls and exploration out of the parent context.
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
It’s funny too because I’ll ask fairly simple things and it’s fine, similarly simple question might spin up a bunch of sub agents and I don’t know why….
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
I recently did a few tests. And always the same prompt has been cached properly.
Cache is usually not shared between agents - they can have different base prompts, tools, and be an entirely different model.
If it's always the same prompt, can't they have it pre-cached globally for all?
I'm pretty sure the system instructions are a function of your environment and not the same universally. That said, there should be a finite number of branches so still cacheable.
For a while everyone was saying sub agents is how you save tokens, use lower quality models with limited context to do simple parts of the job after a smart planning agent has put it all in place. Is that no longer true or is this just the result of sub agent being used at the wrong time?
No, you can definitely configure low cost search and apply subagents. CC and Codex do not. Not sure if this is to improve the reliability of their subagents, or just a play to increase user consumption.
This is why I happily use Codex.
I run it basically 24/7 on a ~500k line repo, and only rarely run out of quota before the end of the week.
My experience with Claude Code was very good until about 2.5 months ago, and then it suddenly turned unbelievably terrible for me.
I have not and will hopefully never look back.
I still have PTSD from how ungodly terrible it was that last week of using it.
> I still have PTSD from how ungodly terrible it was
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
They have Coder PTSD or CPTSD.... Is that a better acronym???
Sorry just teasing.
This has been my experience as well. Something happened 2-3 months ago with Claude Code. It got slower, starting spinning and getting stuck more and more. I gave codex another shot out of my Claude frustrations, and have never looked back again.
Just tried Claude Code yesterday, and nope, it's the same old bad.
Can you be more specific about what “unbelievably terrible” means?
Spawning a bunch of agents seems to happen randomly. I almost never want this.
Such is the nature of tool use
I think there's some setting to restrict the number of them, or maybe turn them off. Doesn't happen for me ~ever and it's not my $$ (work) so I haven't really looked at it much.
And in my experience the sub agent performance is usually worse than just a single agent.
I find it useful for code reviews (spawn a subagent with minimal/no context to review X commit). Of course, this is more or less a shortcut that could be done with a seperate agent. Another use is multiple reviews at once if tokens are not an issue, with seperate "personas" or focuses. As far as implementation goes I have not seen any major usecase.
> What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
> Probably because the general purpose subagents inherit the parent model
only if you don't specify which model should be used
for subagents to be cheap/effective, you have to specify the size of those subagents; i.e. right now by default 5.6-sol spawns many 5.6-sol subagents. 5.4-mini as subagent saves me tons of tokens. 5.6-sol audits the work before accepting it, so there's not really a quality issue.
Subagents with a fat tailed latency distribution completely masks the trough filling that puts the most downwards pressure on per-token COGS.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
It's in the best interest for AI companies to gobble up tokens. I feel like every new release - Fable, etc - is just a way to extract more tokens/money.
Did it deploy five AWS m8g.12xlarge instances?
--disallowedTools Task
My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions. This is supported by the fact that they won't let you use your sub on a different coding agent. I use pi btw.
Once I realized that Anthropic is a token merchant, I start to understand Anthropic’s decision more. They are always finding reasons for you to use more tokens through them unless the users revolt or demand some guardrails.
I've done a couple side by sides on web chat with the same prompt on Opus 4.6, 4.7, and 4.8 and the output gets longer/more verbose on version increment. The enerr variants are definitely much wordier.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
I think both things can be true: new models benchmark higher and eat more tokens.
I bailed on Anthropic the moment they started blocking alternative harnesses like pi on their subscription plans.
Seems unlikely they'd be this dumb. The way to get us to use more tokens is to make those tokens more useful, not less. Anthropic is full of people (including higher-ups) who know this.
But it is much much simpler to make it consume more tokens.
It’s like that saying “What Andy giveth, Bill taketh away”, but in this case it is one company.
There is definitely a conflict of interest.
But they gave us double the tokens! Then a limited time more usage! Then even more tokens "off peak" times! Then some new model released but apparently it inherently used 1.69x tokens! Then Fable is here but "it uses much more usage". But only until ~~the US banned it~~ ~~7th July~~ ~~19th July~~ who even knows.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
now reealize that LLMs are trained to produce tokens and like the halting problem, cant be trained not to produce tokens and youll realiE the AI labs are the perfect essential capitalist and like cancer, will keep growing useless tokens until it kills its host.
no amount of alignment will stop aomeone drom just shutting up.
I thought I read somewhere that according to filings for going public, subscription revenue is tiny… like 5%.
Edit: consumer Claude subs are the 5%. I’d bet most all of CC subs lump in under enterprise.
The vast majority of my company's enterprise plan use is through Claude Code even though we have access to the API and could be using OpenCode instead.
I don't fully agree with the premise that they intentionally increase system prompts, but the enterprise plan usage is going to make that a huge income for Anthropic.
You're making the opposite argument. Anthropic is incentivized to use less tokens in Claude Code because people are paying a fixed monthly fee for subscriptions.
Nope, that’s not true, because they want you to pay for the higher subscription bracket.
Generally, companies with >150 people can’t use subs. So yeah, it’s mostly a funnel for devs/small companies to eventually vet for the product and convince their enterprise to use it as well.
Can confirm — they got me paying $100/mo this way.
Also I think it’s well known that OpenAI is the much less expensive option (in tokens and $$). For the same $20 you get a lot more mileage.
Curious if folks have strong opinions about the overall UX of OpenCode vs CC…
For me as well, at least this month to use more of Fable. We'll see if they extend Fable access because of people like me.
Well since what you get for your subscription is unknown it would be trivial to get that result without burning tokens.
Especially since compute is such a scarce resource.
If they wanted to play games with sub tiers they would just change the rate limits rather than wasting inference.
Flip side is customer psychology. Choosing a more expensive tier leaves better emotion.
Also i doubt there was jira ticket with “make llm more verbose”, rather ticket with “bug makes llms too verbose” gets prioritised taking revenue impact into account.
Enterprise users are not paying a fixed fee, though
Yeah, I strongly recommend against Claude Enterprise, it is ridiculously expensive and hard to control costs.
> I use pi btw
Not sure if intentionally meant as a reference, but it gives "I use Arch btw" vibes.
Pi is one of the ways out of this problem (OpenCode another) so I took it as an intentional reference as it is highly relevant. I also use Pi as my daily driver and I think it's a wise choice to figure out how to decouple yourself from lab-specific harnesses that you have little control or observability over.
the amount of system prompt wastage going on in orgs is insane. we identified 400k in annual burn for zero value in just one section of our large company.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
> This is supported by the fact that they won't let you use your sub on a different coding agent
I mean, that's a very weak argument? Isn't a much more plausible explanation that with your tooling you'll have more of a lock-in than with just your model?
Neither is mutually exclusive.
They get lock-in, and through that lock-in are more effectively able to inflate token usage.
UPDATE:
After reading PUSH_AX's valid comment: ``` This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000 Are we measuring and caring about the right thing? ``` We will update the post to include:
1) A more in-depth task. 2) Qualitative results comparison. 3) As soon as possible, a reproduction of the inputs and outputs.
Thanks, I'm looking forward to this!
I wonder if a lot of the 33k is context, like from recent conversations.
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls:
https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-...
Tokenflation seems very real: the number of tokens consumed by simple tasks keeps increasing.
I often find myself annoyed when Opus fixes a typo in a comment and decides to run tests, lints and whenever else it can find to run. Often it will start by stashing current changes just to preemptively check if all tests were passing before. And I can blame myself a bit because my rules do say: verify all changes with tests. But as there is that I in AI that is hyped which you’d think means it knows not to put tomatoes into fruit salad …
> [..] my rules do say: verify all changes with tests
I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint.
If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
Following rules like "verify all changes with tests" down to a tee is usually a desirable trait in LLMs. Personally I'd leave that behavior there (just like with humans for some tasks like aviation you have them go through checklists even if some stuff you can infer is not needed). But otherwise just make it "always run tests unless you're absolutely sure they can be skipped".
Add "... unless the changes are trivial, docs-only, or typo fixes" to the "always verify with tests" instruction and see how that does
> prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
Recently switched to Codex after 6m in Claude. Codex seems more open, it’s easier to follow what the model is doing and the approvals have a better UX. Overall, it just feels more transparent. Cost of switching was close to 0.
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I use both now and agree they're basically interchangeable.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
And OpenAI didn't try to silently degrade performance of their top model if its (extremely sensitive) safety sensors went off ...
Anthropic is the silver lining keeping p(doom) below 1.0
What settings have you tried since it "became more opaque"? They've got a lot more settings now.
CC went from sane defaults in late 2025 to feature scope creep early 2026. So more features might be good, but sounds like an ick for me. But I have zero prestige, I might switch back.
And pi agent is even less.
The entire agent system prompt can be seen here:
https://github.com/earendil-works/pi/blob/main/packages%2Fco...
Maybe related to this minimalism, Pi doesn't come with most of the tools an LLM needs to function efficiently or effectively. I get that a blank slate is the paradigm, and you can add whatever you want, but it's too blank IMO.
Oh-my-pi has more tools than claude and opencode, and uses them much more efficiently. my favorites are /collab and the gortex mcp
I have a functional Pi config, mostly self-made (it has everything I want, incl. subagents, web search, a /btw command, and other misc. addons), and my system prompt is ~3k.
Would you mind sharing?
It's easy to add using plugins.
What do you miss? I ask because I do some heavy work with pi + GLM 5.2 (using opencode Go subscription) and my workflow is plan -> implement.
> It's easy to add using plugins.
Sure, but you have to add almost everything, no? It deliberately only comes with read, write, edit, and bash. My point wasn't that you can't add stuff, but that I'd just rather use an harness that's a bit more full featured from the start.
(Pi is a bit like old 3D printing where fettling the printer to work is a central part of the hobby. I'd rather just buy a Prusa.)
I'd like to understand what features you're referring to that are missing from base-install Pi CLI.
Read through it an I'm curious whether setting the date and cmd on every system prompt call will cause the cache to invalidate.
I guess the cache would only be invalid if the day changed or the root directory, which would technically happen infrequently enough.
I was here looking for this comment = )
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
Exactly! I just vibe coded (with GPT Sol and Claude whatever-number) my own agent, it's trivial to add now any feature I want - simply ask more powerful model to do it for you. I am happy with end result, however it looks indeed these tools are trained to increase token count - they do quite stupid token-spending steps while making code, but the code itself is also a bit weird - it's like they intentionally do code which is hard to modify on your own without using exactly those authoring models. Interestingly, when I am using DeepSeek with OpenCode, I don't see that - it understands my intent well enough and overall code quality is not bad. I recently switched to local Gemma 4, and I often switch (in opencode) to just that less powerful model, because it understands my intent and has enough skills to provide good quality solution although it's rather for small size projects, and for not coding from scratch, but it's also free and private. It feels slower than any big cloud model, so my model switching is probably most quickest path to robust end result :)
I wrote my own harness in Emacs and it’s completely ridiculous how well it works. Auto-compact is the only missing feature on my list. Claude‘s approach, if I understand it correctly, invalidates a lot of cached context, and I‘m thinking about a more cache-friendly strategy.
This is a truly underrated approach IMO
Any tips on how to get started?
At a minimum, you need an inference endpoint: either cloud or local.
If going local, llama.cpp is going to be the more beginner friendly local inference engine that supports more processor types (AMD GPUs, Intel GPUs, CPUs, anything that supports Vulkan, not just Nvidia). LM Studio is a nice wrapper for this if you'd rather avoid cloning repo and compiling yourself, provided you don't mind closed source software; it's much less enshittified than Ollama.
If going local, you will also need model weights in the right format for your inference engine, and with a model that can fit on your hardware. This is going to be .GGUF files if you're using llama.cpp or a wrapper for it like LM Studio.
From there, pick a language, go look up the OpenAI /chat/completions API format (or Anthropic's "Responses" API format), create a DS or array or slice to store messages, and build a loop that accepts user input, formats it according to the API format, sends it to the inference server, retrieves and parses the response, adds the response to the DS/array/slice, and repeat.
There's a lot more beyond this - tool calling, other API formats (optionally), MCP servers, transport layers besides terminal stdin/stdout, permission models, starting with a system message, clearing your message stack correctly (hint: don't reset it mid tool-call), message compaction, web searching and page fetching, semantic search RAG over embeddings, memory layers - way too much to cover exhaustively in a single message.
Why don't we have some equivalent of "fork" if we are talking the same context and tokens, you'd think that could all just be loaded into the gpu.
I am forced to use cloude code at work but a good solution is to just use --system-prompt "" and be done with it. I wish they allowed for other harnesses.
> --system-prompt ""
Doesn't the model need at least a basic system prompt to understand what tools are available?
The flag name is overloaded. It won't affect the tools available, just the other system instructions.
Yep, have been using this for a long time now. No idea why everyone doesn’t.
Does it have any negative impact? If not, I’m not sure why this wouldn’t be the default behavior. It feels like Anthropic is just putting their foot on the scale to drive up costs or for the enterprise, or push consumers to higher subscription tiers.
I didn’t know you could do this. Is there any analysis of the impact, before and after? I’d love to see some charts of efficacy in real world usage.
It shows up in /context, but never spend time validating it much. Some people run a proxy to modify their messages.
Do you start Claude with this option? Or do you send this with every prompt?
yep I pass it to the CLI, I also pass --model
A harness is a part of the intelligence stack. It's no longer about raw access to the model
Also, I have seriously used most harnesses - One feels like it's being built in a place that truly understands AI and where agentic engineering is headed. You might not like it, but peak performance exists in CC when it comes to orchestration of bulk parallel work / subagents. The open source agents are catching up or accell in different way (Im preferable to pi.dev), but I'm not sure they're architecting orchestration the right why.
Nothing about the time taken to complete the task? Users are definitely sensitive to time, not only token consumption.
Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow.
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
This is interesting, because if I start a fresh session of Claude Code right now and run /context, I see the following:
4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc.
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
Claude Code is not just a harness. It is a different product. You pick the smallest subscription that allows you to do your work. My “multiplier” on a $100 subscription is 5+.
If you’re using API, on the other hand, there is absolutely no reason to use Claude Code, or Codex.
pi sends 1k (or less) -> https://github.com/earendil-works/pi/blob/main/packages/codi...
My $20 sub using gpt 5.6 sol thinking-off lasts for hours using pi.
Why turn thinking off? I mean, yes, it uses less tokens, but you're using the best model OpenAI offers, but then making it as dumb as cheaper models.
We are yet to try Pi!
This is not a “discovery” in any sense - both Claude Code and Opencode display token usage clearly after starting a chat.
Mine sends even less - https://maki.sh
Nice!
> When context gets too long, maki compacts history automatically: strips images, thinking blocks, and summarizes older turns.
Don’t the summaries of older turns effectively invalidate the context cache, such that you consume less tokens but more expensive tokens?
Only once per compaction
This is all heading in the right direction. Much of AI coding feels magical. But when the costs begin to accrue we start asking questions. We dig into it and try to understand what's going on. I can't help but feel Anthropic is "token maxing" from its side: it controls the levers and with every version upgrade it can build in its own token growth almost unbeknownst to the user. This actually harms it on the long run because it necessitates a cheaper option.
Is it not a conflict of interest for a model provider to supply the harness? They are not motivated to minimize your costs.
They sort of are, in that they want subscription users to have clients that behave well with the KV cache etc.
If you don't use a subscription, and pay per token instead, you can easily move to another harness.
Claude Code sending 33k tokens before reading the prompt is the AI equivalent of a consultant who bills you for the time spent reading your email before they even open it.
Well, I have to open the lid on my computer and remember my password, no?
The reasoning built into the models matter so much too. I recently swapped my Qwen3.6 27B to ThinkingLabs’ fine tune and it does what it publishes. I cut my token usage in half, which is a big deal since I only get ~20 TPS for token generation.
This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000
Are we measuring and caring about the right thing?
Anecdotally, the results from OpenCode + Claude appear to be the same if not better for our uses over the past year.
I think this doesn't mean much; the axes that matter are intelligence x dollars x time; tokens by themselves mean nothing.
With Fable being per token instead of on the subs (unless they changed it again?), I decided to test Claude code on OpenRouter where I had some credits, with Opus 4.8 and Fable 5.
I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
As for context size and harnesses I did make a trivial bash agent based on this "agent in 50 lines" tutorial[0] recently, and found that for trivial work, it was about an order of magnitude cheaper and faster.
I haven't tested it on anything bigger but it doesn't seem to do the kind of proactive testing, that they do in bigger harnesses.
Codex at least has a system prompt that tells it not to consider a feature a complete until it has verified it. I'm not sure about Claude Code.
I suppose I could add that one line to the prompt, and it would get me much closer to agi :) I think Fable does this proactively even without a prompt, but I haven't tested that yet.
If Fable in my own harness is significantly cheaper than Claude Code, that would be very appealing. (I could actually afford to use it for most things!) But I think most of the cost comes from the testing it does. So we'll have to see.
[0] https://minimal-agent.com/
Fable's subscription inclusion theoretically ends EOD today. Anthropic put a wishy-washy "if we have capacity we'll continue it" thing, and given how competitive GPT 5.6 Sol is, and it is included in OpenAI's subscription, I fully expect Anthropic to extend Fable or they will have a serious exodus on their hands.
Competition is good.
Anthropic have extended Fable access again to July 19. The notice should pop up in your Claude Code now when you start a new session (also announced on the ClaudeDevs X account first).
Ah, thanks. It's been hard to plan around these last-minute changes. I rushed to implementation on a spec I should have spent more time on because of the looming deadline.
I've been trying various harnesses like Pi, OpenCode, Qwen Code, and Nanocoder. A common problem I keep running into is failed tool calls, regardless of the model. What is the best harness and on-device model combination right now?
> and on-device model combination right now
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
I'm looking for something that runs on an M5 Macbook Pro with 48 GB of unified memory.
You can't afford the best model. What are your specs and what models + quants have you tried?
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.
Pi.dev requires some plugins to work well. Using Qwen3.6-27B/35B locally at Q8, I was quite frustrated with failed tool calls and tried many things.
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
2. https://pi.dev/packages/@aboutlo/pi-smart-edit - white-space-tolerant edits, as Qwen would sometimes add a fifth space to a four space indent
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
https://archive.is/O2BFs
I feel like this article isn't saying much. Even with tools disabled, Claude Code still has a crap load of commands and other things that Claude (the model) should know the availability of since it's optimized for them. All of that has to be disabled if this is to be a real harness comparison. And of course the system prompt can be completely replaced, making it a no-brainer to use a more minimal prompt similar to OpenCode. And beyond that nothing else really matters because the rest (cache behavior, etc) lies with the provider's platform, not the harness.
Grok 4.5 is really fast, has more usage at $10/month than $20/month Claude pro, and Opus-level. Claude pro feels like a demo.
Claude is much better in OpenCode then in Claude Code, OpenCode is just better than Claude Code. Claude Code feels like a complete mess to use comparatively.
that makes sense, claude code actually does inflates token usage
not even surprised
Why don't people fix their costs (rent a gpu) and just write their own harness (about 200 lines of code).
Supposed to be hacker news and half the posts are like "this harness steals this" like it cant be avoided.
These API costs are mad.
GLM isn't good enough yet.
It pays to be marginally ahead of people stuck on open models.
Sorry for asking here, but nobody seems to know.
If I self host a local model is there some way to make Android studio not time out after 10 minutes?
I recommend that Opencode users try Dynamic Context Pruning as well: https://github.com/Opencode-DCP/opencode-dynamic-context-pru...
It works great for long-horizon tasks, and feels like it saves a boatload of tokens.
The Sleev (the project has been renamed to make a startup) creator was shilling their project in the OpenCode Discord. That person is very convinced they have something that no one has ever built before. They focused on token reduction without any real evals for capability impacts.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
It's definitely not unprecedented, but the plugin version is useful. Sleev seems like a nothingburger, I'm happy with the results I get from DCP already.
No surprise, I've noticed that "agents", not only CC (I am using Copilot) are trying to be "clever", searching for a lot of data. This is good for LLM providers as this eats a lot of tokens.
OpenAI, to their credit, seems to be focusing pretty heavily on token efficiency in GPT 5.5 and beyond.
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
Given they're incentivized to increase token use, what guarantees that higher token use improves the effectiveness of the agent and isn't just artificial padding?
Well, nothing really. But I assume there can be some benefits to modifying context. For example, updating file contents or marking them as modified, summarization, injecting additional information, removing irrelevant tool call results, etc.
Is there evidence that it is actually a better agent though?
There’s evidence it’s a worse agent actually. I’m just saying in theory.
So? it doesnt matter, after the first turn it's cached. We are probably talking about single digit cents.
> Claude Code 2.1.207 and OpenCode 1.17.18, both pinned to claude-sonnet-4-5
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Model:
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.