> We speedrun SWE-bench Verified by training from scratch on agentic trajectories. Speedrunning is strikingly effective: our $60 run achieves 5.0% pass@1, matching Claude 2, the SOTA at SWE-bench's release. With $1000 of train compute, our speedrun reaches 11.0% pass@1, outperforming Claude 3 Haiku (7.2%).
nanoswe.com/blog.html
> We speedrun SWE-bench Verified by training from scratch on agentic trajectories. Speedrunning is strikingly effective: our $60 run achieves 5.0% pass@1, matching Claude 2, the SOTA at SWE-bench's release. With $1000 of train compute, our speedrun reaches 11.0% pass@1, outperforming Claude 3 Haiku (7.2%). nanoswe.com/blog.html