From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..
It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.
> I’m not willing to spend the time digging through this to figure out what’s needed
That's the neat part. You don't... need to do this yourself nowadays. AI agent is perfectly capable to figure out any new open source thing that pops up daily now. You can just give it the link to the repo and say "Set this up for me. I wanna play with it."
From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..
The skills can be found here:
https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...
Question that was left unanswered:
How much longer does a 4x DeepSkeep loop took compared to Opus?
Seems interesting, but buried under a mountain of written slop.
It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.
You could literally send it to an ai
> I’m not willing to spend the time digging through this to figure out what’s needed
That's the neat part. You don't... need to do this yourself nowadays. AI agent is perfectly capable to figure out any new open source thing that pops up daily now. You can just give it the link to the repo and say "Set this up for me. I wanna play with it."
until you realize people are putting malicous instructions in the projects now, at least a few anti-Ai coders see this as an act of defiance