You may find this paper interesting, I came across it when I was trying to figure out why SimpleQA scores are higher in base models than instruct versions:
1) is definitely made worse by post-training, which kinda makes sense, as it’s optimising the weights away from predicting text in language and towards solving binary task completed/not gates
The weird thing is I kinda agree that it feels like I didn’t notice this with earlier models, tho maybe it just took time to start noticing this?
From most used to least used: Claude, DeepSeek, Kimi, Gemini
With claude, it's been writing insanely unreadable comments that read more like poetry than technical writing. I've been wondering if this is a model-family specific problem or a general problem.
You may find this paper interesting, I came across it when I was trying to figure out why SimpleQA scores are higher in base models than instruct versions:
https://arxiv.org/html/2406.14972v1
1) is definitely made worse by post-training, which kinda makes sense, as it’s optimising the weights away from predicting text in language and towards solving binary task completed/not gates
The weird thing is I kinda agree that it feels like I didn’t notice this with earlier models, tho maybe it just took time to start noticing this?
How many different model families have you tried? I'm really liking Kimi K3.1, including technical writing
My hunch as well is the behaviors you describe are a product of training towards running human free for longer. I'd prefer something different
From most used to least used: Claude, DeepSeek, Kimi, Gemini
With claude, it's been writing insanely unreadable comments that read more like poetry than technical writing. I've been wondering if this is a model-family specific problem or a general problem.
[dead]