This led to the natural question: can gzip do language modeling? (...). Here’s some real, unedited output after priming it on tiny Shakespeare:
gzipt --corpus data/tinyshakespeare.txt --prompt $'MENENIUS:\n' --length 200
MENENIUS:
'Though all at once canq
MARCIUS:
Pray now, nocamest thou to a morsel.
LARTIUS:
Hence, and
I' the end admire, where G
again; and after it ag .
Now thinking back, what's missing so that gzip could unwind the correct body of work from Shakespeare is just a correct sequence of bytes. One way to arrive at this is by just getting the body of work and doing the inverse, compressing it to get that golden sequence of bytes.
The other is what thinking does, it tries to predict the missing sequence of tokens from a high entropy source, the prompt, in order to increase the likelihood of correctly decompressing the desired results from its weights.
How can things be compressed without losing information or structure?
Like for text, what would that involve? How do you compress a string or multi-line string without losing information and hopefully structure (paragraphs, would it be like replacing periods and the following space with just sticking the starting capitalized letter of the following word to the previous sentence's last letter and when it decompresses theres some kind of note that converts that back into the. First letter of the next sentence
How can things be compressed without losing information or structure?
Because the initial content is rarely the most efficient representation, so it's possible to store fewer bytes that can deterministically be converted into the original.
Like for text, what would that involve?
Most compression algos don't care what information you're compressing. All they see (all they need to see) is bytes. It ends up being way more sophisticated than removing repeated periods and whitespace.
Like if you had eight boxes of loose lego, simply shuffling around the boxes wouldn't give you much in the way of reducing the space the legos take up. but if you took the legos (bytes) themselves out of the boxes, you end up saving a lot more space.
It reminds me of "At the time we drew boxes labeled 'perception', 'cognition' with arrows between them." An imprecise quote that I can't place.
I guess my box labelled 'subconsciousness' is trying to say that low-level mechanisms that give rise to the observed cognitive phenomena might have nothing to do with neat boxes.
McDermott's A Critique of Pure Reason pretty much captured all of the misgivings I had about "Good Old-Fashioned Artificial Intelligence", which was slightly unfortunate as I was trying to complete a PhD in that very area at the time (around 1990 or so...)
No - there were lots of other interesting things going on in the early 90s that I managed to pivot to. Never regretted not completing as I had given up any desire to work in academia by that point.
Interestingly that is not what we got, but maybe we should loop at architectures like this again? The JEPA loop is interesting, but might fail for the in-flexibility of the component ordering
How relevant is this fast/slow thinking thing with regards to current frontier models?
I know a large organization who's built their AI framework completely around this concept, and I feel that it's not really meaningful concept with the capabilities of current models.
That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language.
It can be said that language is a tool for the serialization (writing) and deserialization (reading) of human ideas. It is also an incredible useful and powerful tool by itself.
This last sentence has been proved true by LLMs themselves.
However, since it is working on the serialized version of ideas, I agree with you in that's not the optimal way to think and something not serialized (maybe world models) can be invented that's better for thinking.
All this in no way diminishes the usefulness of language and of automated language generation.
> That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language
Do we? I just learned from a speaker[1] that we literally need words to recognize emotions. People who have a poor vocabulary have lower emotional intelligence because without being able to attach a word to an emotion, the brain is unable to recognize & process it.
[1] Dude seemed to be knowledgeable about the subject. He's a specialized trainer, should be educated in this exact field. So hopefully I'm not lying to anyone here :)
No clue what’s the consensus on this but my internal mental model is absolutely that LLM AI is pure fast mode, no slow mode. The “reasoning” loops are an attempt to mimic the slow mode but ultimately it doesn’t really work. I’m curious about the recent maths advances though, they seem to possibly challenge this.
In terms of making an LLM faster but not in terms of meta-cognition. System 1 thinking as defined by Kahneman doesn’t have 100000x more compute than System 2, it is actually the opposite. That completely contradicts your claim
Structurally speaking we learn nothing like AI, we don't use vast amounts of information to pick up completely new skills. We also make decisions by using prior knowledge and emotions.The latter part is important, Thinking fast and slow cannot operate in a world of AIs as they stand today unless we are willing to grant them rights — because you have to teach them to make decisions based on all kinds of emotions — which is tricky at best.
It's indeed a terrible idea - as in, it's great. You get benefits of cross-marketing: you ride on a popularity of a well-known book, and as you also drive more sales of it, even if you don't have a deal and don't benefit from that directly, you strengthen the loop and solidify your brand.
This choice doesn't really constrain what the organization can do, either. Pop-sci books have plenty of wiggle room in interpretation, and afford a lot of "you're holding it wrong" dismissals of criticism, that with a bit of clever copywriting, the organization can do absolutely anything and still claim it's embodying the framework/theory of the book.
It looks like a lot like how data bases query optimizers work, with the exception that in the paper there is also a learning/memory component that conditions the evaluation of the answer provided by the first model.
This is still a great paper, but it's missing the second axis of the quadric -- if the only two options are thinking fast or thinking about thinking, that leaves no room for thinking slow yet deliberately, AKA selfconsciousness. See https://www.gutenberg.org/cache/epub/4280/pg4280-images.html for details
I do wonder if any of these folks ever got a chance to try this at one of the big labs, tho...
This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
* The person who posted it likely posted it not as an out-of-date paper but as an interesting idea. Your comment ignores the idea and focuses on what you're calling its out-of-dateness.
* You say "this has been solved" without defining what "this" is.
* Your description of the solution -- different effort levels -- seems to indicate that you misunderstand the idea that the paper is proposing. If I understand their proposal, it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model and past experience. "Effort" isn't so much the issue as types of effort using different systems, modeled specifically after Kahneman's idea of fast and slow thinking.
* The title is an allusion to a book by Daniel Kahneman. The brisk dismissal without acknowledging the idea or the history doesn't leave a good impression, even if I'm mistaken and you're right.
In short, Hacker News readers tend to reward depth and detail (the FAQ specifically encourages thoughtful contributions and explicitly discourages dismissal). Your comment doesn't provide them, and it appears to make a mistake that further undermines its value as a contribution to discussion.
> For the first time, GPT‑5.1 Instant can use adaptive reasoning to decide when to think before responding to more challenging questions, resulting in more thorough and accurate answers, while still responding quickly. This is reflected in significant improvements on math and coding evaluations like AIME 2025 and Codeforces.
> This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
Tell me you didn't read Daniel Khaneman's book without telling me you didn't read Daniel Khaneman's book.
> I know what system 1 and 2 is. But this has already been solved using same model.
No, it hasn't. Maybe you have a different definition of System 1 and System 2. I last read the book well over a decade ago (2011, maybe? 2012?), but System 1 and System 2 are different systems. IOW, System 2 is not a more computational version of System 1.
The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
In computery terms, System 1 runs in O(1) time, System 2 runs in O(log n) (or maybe just O(n)) time.
This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer"). We don't have LLMs that do that. We have System 2 - run in O(log n) time and produce an answer.
> The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
No, system 2 is the emergent capability to reason and increase the space of places to find the answer. Forget the paper's proposal, and look at the problem it is trying to solve. Ability to give quick answers, ability to give thought out answers, and the ability to know when to choose what. Adaptive reasoning does all three.
> This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer").
No, I don't think we humans use o(1) to for understanding 1000 tokens or 2 tokens. I simply don't think that's the case. There's a new model called "Jev" and it is literally named System 1 (from the book) and even it is billed per input token.
There was this post a few days ago https://news.ycombinator.com/item?id=49797323
It had this to say in the linked post:
Now thinking back, what's missing so that gzip could unwind the correct body of work from Shakespeare is just a correct sequence of bytes. One way to arrive at this is by just getting the body of work and doing the inverse, compressing it to get that golden sequence of bytes.The other is what thinking does, it tries to predict the missing sequence of tokens from a high entropy source, the prompt, in order to increase the likelihood of correctly decompressing the desired results from its weights.
How can things be compressed without losing information or structure?
Like for text, what would that involve? How do you compress a string or multi-line string without losing information and hopefully structure (paragraphs, would it be like replacing periods and the following space with just sticking the starting capitalized letter of the following word to the previous sentence's last letter and when it decompresses theres some kind of note that converts that back into the. First letter of the next sentence
You analyze the frequency of combinations of bytes, then replace those with high frequency with pointers to a single instance.
Like if you had eight boxes of loose lego, simply shuffling around the boxes wouldn't give you much in the way of reducing the space the legos take up. but if you took the legos (bytes) themselves out of the boxes, you end up saving a lot more space.
It reminds me of "At the time we drew boxes labeled 'perception', 'cognition' with arrows between them." An imprecise quote that I can't place.
I guess my box labelled 'subconsciousness' is trying to say that low-level mechanisms that give rise to the observed cognitive phenomena might have nothing to do with neat boxes.
You mean "Artificial Intelligence meets Natural Stupidity" by Drew McDermott
https://dl.acm.org/doi/pdf/10.1145/1045339.1045340
McDermott's A Critique of Pure Reason pretty much captured all of the misgivings I had about "Good Old-Fashioned Artificial Intelligence", which was slightly unfortunate as I was trying to complete a PhD in that very area at the time (around 1990 or so...)
Did you complete it?
No - there were lots of other interesting things going on in the early 90s that I managed to pivot to. Never regretted not completing as I had given up any desire to work in academia by that point.
Donald Broadbent drew a lot of boxes in the 50s/60s (https://en.wikipedia.org/wiki/Broadbent's_filter_model_of_at...). There have been plenty of critics of this tendency, I recall some calling it 'boxology' rather than psychology.
Interestingly that is not what we got, but maybe we should loop at architectures like this again? The JEPA loop is interesting, but might fail for the in-flexibility of the component ordering
How relevant is this fast/slow thinking thing with regards to current frontier models?
I know a large organization who's built their AI framework completely around this concept, and I feel that it's not really meaningful concept with the capabilities of current models.
That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language. It can be said that language is a tool for the serialization (writing) and deserialization (reading) of human ideas. It is also an incredible useful and powerful tool by itself. This last sentence has been proved true by LLMs themselves. However, since it is working on the serialized version of ideas, I agree with you in that's not the optimal way to think and something not serialized (maybe world models) can be invented that's better for thinking. All this in no way diminishes the usefulness of language and of automated language generation.
> That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language
Do we? I just learned from a speaker[1] that we literally need words to recognize emotions. People who have a poor vocabulary have lower emotional intelligence because without being able to attach a word to an emotion, the brain is unable to recognize & process it.
[1] Dude seemed to be knowledgeable about the subject. He's a specialized trainer, should be educated in this exact field. So hopefully I'm not lying to anyone here :)
No clue what’s the consensus on this but my internal mental model is absolutely that LLM AI is pure fast mode, no slow mode. The “reasoning” loops are an attempt to mimic the slow mode but ultimately it doesn’t really work. I’m curious about the recent maths advances though, they seem to possibly challenge this.
You can ask a model for output directly and stop, or you can recursively ask it to keep refining the output.
That seems to fit the fast vs slow model of human thought reasonably well.
> You can ask a model for output directly and stop
That’s still several orders of magnitudes too slow to fit fast vs slow. Think of 30ms vs 3-4 seconds to get an idea of what we’re talking about here
That’s a function of the amount of processing power involved not the underlying architecture of decision making.
In terms of making an LLM faster but not in terms of meta-cognition. System 1 thinking as defined by Kahneman doesn’t have 100000x more compute than System 2, it is actually the opposite. That completely contradicts your claim
The ratio between a single pass and multiple passes is unchanged when you throw more processing power at both.
In a human 30ms vs 3-4 seconds is a 1:100 ratio. Single vs multiple passes with an LLM varies but a 1:100 ratio isn’t unrealistic.
Structurally speaking we learn nothing like AI, we don't use vast amounts of information to pick up completely new skills. We also make decisions by using prior knowledge and emotions.The latter part is important, Thinking fast and slow cannot operate in a world of AIs as they stand today unless we are willing to grant them rights — because you have to teach them to make decisions based on all kinds of emotions — which is tricky at best.
> we don't use vast amounts of information to pick up completely new skills.
Except, we do.
Seems like a terrible idea in the first place to build an entire organization around a single pop-sci book, but that’s just me.
It's indeed a terrible idea - as in, it's great. You get benefits of cross-marketing: you ride on a popularity of a well-known book, and as you also drive more sales of it, even if you don't have a deal and don't benefit from that directly, you strengthen the loop and solidify your brand.
This choice doesn't really constrain what the organization can do, either. Pop-sci books have plenty of wiggle room in interpretation, and afford a lot of "you're holding it wrong" dismissals of criticism, that with a bit of clever copywriting, the organization can do absolutely anything and still claim it's embodying the framework/theory of the book.
It looks like a lot like how data bases query optimizers work, with the exception that in the paper there is also a learning/memory component that conditions the evaluation of the answer provided by the first model.
Given how LLMs access compressed knowledge from their model weights, the similarities make sense
At least this is written before ChatGPT.
Trying to get LLMs to 'think about their thinking' is my daily struggle. This paper nails why it's so critical.
If I recall correctly, all that fast and slow business has been debunked as yet more non-replicable pop psychology.
I shouldn't be surprised that it shows up in a screed on AI
This is still a great paper, but it's missing the second axis of the quadric -- if the only two options are thinking fast or thinking about thinking, that leaves no room for thinking slow yet deliberately, AKA selfconsciousness. See https://www.gutenberg.org/cache/epub/4280/pg4280-images.html for details
I do wonder if any of these folks ever got a chance to try this at one of the big labs, tho...
Isn't system 2 by itself semantically means thinking by thinking
> submitted Oct 5 2021
(In case people miss that before discussion)
Updated, thanks!
This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
edit: why is this downvoted?
It's being downvoted, I think, for a few reasons:
* The person who posted it likely posted it not as an out-of-date paper but as an interesting idea. Your comment ignores the idea and focuses on what you're calling its out-of-dateness.
* You say "this has been solved" without defining what "this" is.
* Your description of the solution -- different effort levels -- seems to indicate that you misunderstand the idea that the paper is proposing. If I understand their proposal, it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model and past experience. "Effort" isn't so much the issue as types of effort using different systems, modeled specifically after Kahneman's idea of fast and slow thinking.
* The title is an allusion to a book by Daniel Kahneman. The brisk dismissal without acknowledging the idea or the history doesn't leave a good impression, even if I'm mistaken and you're right.
In short, Hacker News readers tend to reward depth and detail (the FAQ specifically encourages thoughtful contributions and explicitly discourages dismissal). Your comment doesn't provide them, and it appears to make a mistake that further undermines its value as a contribution to discussion.
> it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model.
do you even know how adaptive reasoning works?
Godspeed to you in your efforts to contribute productively to Hacker News threads.
> For the first time, GPT‑5.1 Instant can use adaptive reasoning to decide when to think before responding to more challenging questions, resulting in more thorough and accurate answers, while still responding quickly. This is reflected in significant improvements on math and coding evaluations like AIME 2025 and Codeforces.
https://openai.com/index/gpt-5-1/
It says literally the thing you wanted from system 2. Its almost exactly that.
This is what you said btw:
"it's that the system itself decides how to reason based on the nature of the problem it faces"
> This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
Tell me you didn't read Daniel Khaneman's book without telling me you didn't read Daniel Khaneman's book.
Asking earnestly, I don’t know what you mean by this reply. I know what system 1 and 2 is. But this has already been solved using same model.
> I know what system 1 and 2 is. But this has already been solved using same model.
No, it hasn't. Maybe you have a different definition of System 1 and System 2. I last read the book well over a decade ago (2011, maybe? 2012?), but System 1 and System 2 are different systems. IOW, System 2 is not a more computational version of System 1.
The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
In computery terms, System 1 runs in O(1) time, System 2 runs in O(log n) (or maybe just O(n)) time.
This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer"). We don't have LLMs that do that. We have System 2 - run in O(log n) time and produce an answer.
System 1 is completely bereft of thought.
> The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
No, system 2 is the emergent capability to reason and increase the space of places to find the answer. Forget the paper's proposal, and look at the problem it is trying to solve. Ability to give quick answers, ability to give thought out answers, and the ability to know when to choose what. Adaptive reasoning does all three.
> This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer").
No, I don't think we humans use o(1) to for understanding 1000 tokens or 2 tokens. I simply don't think that's the case. There's a new model called "Jev" and it is literally named System 1 (from the book) and even it is billed per input token.
Surely that is obvious
2021. Please remember the rule of HN to add the year if it’s not actual.
Good luck linking to that rule.