Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed:
- they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use.
- they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine.
- they are going to get a separate dedicated evaluator specifically for bioweapon risks
Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed:
- they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use.
- they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine.
- they are going to get a separate dedicated evaluator specifically for bioweapon risks
Feels like a good thing to give an update on right before you launch a new version of Fable.
I don't need anything better than Opus 5.5. Maybe if you wanna make more maths discoveries you'd need something else.
640k
480k
Well, it was nice to have a functional models for a bit. Shame it didn't last long.
???
Fable 6 gazes into your soul and gives you not what you ask for, but what you deserve.
!