Terminal bench 2.1: Same score, 16% cheaper run Lemoncrow

3 points | by pankaj4u4m 11 hours ago

2 comments

LemonCrow (opus-4-8) vs Baseline (n=445):

  Avg turns/trial:       13.74  vs  16.82   (-18.3%)
  Avg duration/trial:    448.8s vs  550.0s  (-18.4%)
  Total wall time:       55.5h  vs  68.0h   (-18.5%)
  Cost(both @ 1hr tier): $279.40 vs $411.58 (-16%)

pankaj4u4m 11 hours ago
[dead]