Shout-out to anthropic for having their models have such a strongly distinct writing style and personality that you can recognize their work instantly! It's quite nice to have such an immediate signal that if I were to proceed, I would spend orders of magnitude more time and effort reading the the text than the person claiming author credit spent writing or even reading it themselves.
Am I missing something or the evals do not compare it to the baseline deepseek-v4-flash? Without a baseline comparison, it is hard to tell what works well and what doesn't
I've been hearing forever that latent reasoning is the next step. But none of the frontier labs seem to incorporate it. Any idea why? It doesn't sound so technically challenging. Sure you loose some interpretability but surely everyone would sacrifice that for a step-up in quality. Or at least offer a latent-reasoning-version of the model.
you lose CoT monitorability which is a big issue since models have become quite powerful and also often deceptive but i do think that efficiency pressure will keep nudging us toward latent reasoning. looped language models are an active research area. but imagine not being able to monitor mythos' thoughts as it's working through a national security need...
LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens.
Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking.
Interpretability is a mixed bag even with trained tools on top of existing models.
My main motivation is to be able to tune the thinking effort without polluting token IO. Inspiration came from the CoLaR paper and other works showing that repeating laters in an LLM got better results.
Cool idea. I didn't understand what causes the bad response on the first query. Does it mean the first response in every new conversation, or just the first served response after startup?
After vLLM startup I observed this on first request only then all further conversations were fine, new and original. Can’t figure out why but seems to be fine after that.
LinkedIn recently added a “seems like AI slop” button. I.e.: independent of downvoting/not interested/flagging as spam/ToS violation, you can say “this is AI slop”. Maybe we need something like it here
The content is unreadable. No comparison to the underlying model. Massive text expansion. Hard to tell if the numbers are real or entirely hallucinated SEO slop.
Tbh I appreciate knowing immediately that no effort was put into writing an article and that no effort is worth being put into reading it. I genuinely (hah) think it's good that different llms have distinct voices when writing. It doesn't bother me when using it for coding because ok that's just how Claude talks, and in the wild seeing folks present claudeslop as their own writing is a really nice really quick quality/effort signal. Adding another step to the prompt doesn't mean whoever generated this report actually out any more real effort into making sure this is any good
In my view, it's not so much the writing style itself as the lack of 'taste'. Text that clearly seems AI-written has a flat level of exuberance that's just exhausting, kind of like a written version of the 'loudness war'.
Without some kind of dynamic range, I find myself having to do a lot of work to infer what points are truly important versus what are at best interesting implementation details.
The article is AI-written as well, with all the "honest problems" and "real weak spots". The code must be AI-generated too, so without a human properly verifying it, I can't take the project seriously. It may just be AI hallucinations congratulating themselves on imaginary achievements.
Shout-out to anthropic for having their models have such a strongly distinct writing style and personality that you can recognize their work instantly! It's quite nice to have such an immediate signal that if I were to proceed, I would spend orders of magnitude more time and effort reading the the text than the person claiming author credit spent writing or even reading it themselves.
Fair critique, ill rewrite (this is just my personal pet project but thought was cool to share here)
You guys are so proud of yourselves lol
Am I missing something or the evals do not compare it to the baseline deepseek-v4-flash? Without a baseline comparison, it is hard to tell what works well and what doesn't
I can do that, main difference seems to be this ablation seems to over think simple prompts, but can do a proper eval (might take 2-3 days)
I really appreciate all the honesty here.
I've been hearing forever that latent reasoning is the next step. But none of the frontier labs seem to incorporate it. Any idea why? It doesn't sound so technically challenging. Sure you loose some interpretability but surely everyone would sacrifice that for a step-up in quality. Or at least offer a latent-reasoning-version of the model.
I am on my own GPUs so thought it be cool to reduce token output. you can charge per latent loop as a token i guess.
I wanted a knob to keep thinking going until it was sure it was done.
you lose CoT monitorability which is a big issue since models have become quite powerful and also often deceptive but i do think that efficiency pressure will keep nudging us toward latent reasoning. looped language models are an active research area. but imagine not being able to monitor mythos' thoughts as it's working through a national security need...
LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens.
Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking.
Interpretability is a mixed bag even with trained tools on top of existing models.
AI2027.com puts this around early 2027 and treats the loss of legible chain-of-thought as a major negative turning point for alignment.
My main motivation is to be able to tune the thinking effort without polluting token IO. Inspiration came from the CoLaR paper and other works showing that repeating laters in an LLM got better results.
Presumably you could decode the chain of thought from the latent representation though.
Cool idea. I didn't understand what causes the bad response on the first query. Does it mean the first response in every new conversation, or just the first served response after startup?
After vLLM startup I observed this on first request only then all further conversations were fine, new and original. Can’t figure out why but seems to be fine after that.
LinkedIn recently added a “seems like AI slop” button. I.e.: independent of downvoting/not interested/flagging as spam/ToS violation, you can say “this is AI slop”. Maybe we need something like it here
Fair criticism, i’ll rewrite it… as this is a pet project and post based on another HN comment.
The content is unreadable. No comparison to the underlying model. Massive text expansion. Hard to tell if the numbers are real or entirely hallucinated SEO slop.
[flagged]
[flagged]
[dead]
[flagged]
Tbh I appreciate knowing immediately that no effort was put into writing an article and that no effort is worth being put into reading it. I genuinely (hah) think it's good that different llms have distinct voices when writing. It doesn't bother me when using it for coding because ok that's just how Claude talks, and in the wild seeing folks present claudeslop as their own writing is a really nice really quick quality/effort signal. Adding another step to the prompt doesn't mean whoever generated this report actually out any more real effort into making sure this is any good
In my view, it's not so much the writing style itself as the lack of 'taste'. Text that clearly seems AI-written has a flat level of exuberance that's just exhausting, kind of like a written version of the 'loudness war'.
Without some kind of dynamic range, I find myself having to do a lot of work to infer what points are truly important versus what are at best interesting implementation details.
You're right to push back on that. The honest truth is that the slop is load-bearing.
[flagged]
The article is AI-written as well, with all the "honest problems" and "real weak spots". The code must be AI-generated too, so without a human properly verifying it, I can't take the project seriously. It may just be AI hallucinations congratulating themselves on imaginary achievements.
As is this whole blog post... Very grating prose