Absolutely fascinating topic but if there ever was a good time to say "we have no clue and should reserve any judgement" it's about consciousness.
How can you rule anything out when you can't even observe/verify the phenomenon from a 3rd person perspective, let alone form a falsifiable theory about how it arises
We can, however, observe it from a first person perspective.
We can make inferences from observations.
Its great when you have a testable falsifiable hypothesis, but knowledge can advance in other ways. That is how we get philosophical ideas, such as the concept of falsifiable hypotheses.
That's not fully true, though. If you shoot something that was conscious and kill it, it stops being conscious. So you can observe/verify the phenomenon from a 3rd-person perspective, but you can't study it objectively.
Furthermore, we know a lot about the brain itself and how impacts/defects can affect the conscious experience.
It is true that we cannot ever experience consciousness outside our subjective experience, but it is surely verifiable/observable from a third-person view (or rather its effects).
Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.
> We really know almost nothing about consciousness
I think this is all that's really relevant here. People make extremely confident statements about this, but objectively, we hardly know anything. I don't even know if anyone other than me is conscious.
To be honest, I'm quite confused about why I am conscious. It seems implausible that a bunch of cells would be able to create one self that experiences consciousness, and yet I do experience consciousness. Given how weird that is, I do not feel that I have any level of trustworthy intuition or knowledge about what else might experience consciousness.
> To be honest, I'm quite confused about why I am conscious.
The words "conscious" and "consciousness" are hypostasized. They name or identify things and thereby establish them even though there is scant tangible evidence they exist. As conscious beings, we have no known tests or measures for consciousness other than the report of beings who claim to be conscious.
That last sentence can, and should, be rewritten as follows to make the point stronger: as _allegedly_ conscious beings, we have no known tests or measures for consciousness other than the report of beings who claim to be conscious.
This includes you, OP. You assert consciousness and while I and everyone who reads your comment has no reason to doubt you, we should. And you should.
Humans do not know what consciousness is and asserting that you or I or they have it is not supported by any evidence except for the claim.
Post Script: In a different universe, 5 years ago before LLMs, I would have made the above comment without further thought. However, even if consciousness does not have a single suite of tests to establish its fact, there are numerous physical and measurable correlates of consciousness that do provide evidence consciousness unites them, whatever consciousness might be. (e.g. "neurological assessments, experiments involving perceptual awareness, and measurements of brain activity under anesthesia or during disorders of consciousness." [0])
In other words, consciousness may not be as hypostasized as all that.
[0] ChatGPT's partial response to my question: "Is the word hypostatize useful to describe a concept or thing that is established but poorly evidenced? For example, the word "consciousness" is traded and used by many people but there are no known tests or measures for consciousness other than the report of beings who claim to be conscious. Therefore, "consciousness" is hypostatized."
I think saying that the way we talk about knowledge in the modern world puts too much weight on describing things quantatively. I think knowledge is still valuable even if it cannot be described quantatively. For example, people knew how to breed dogs for millenia before DNA was discovered, and football players can kick a ball accurately without understanding the quantative aerodynamics of the ball. Folk science is almost always the starting point of rigorous science, for example, people were classifying plants and animals before there was a concept of species.
So I think when it comes to conciousness, instead of saying we don't know, it'd be better to start with what we know intuitively through folk science and then try to add more rigor. Even if our intuitions about conciousness are not rigorous, they are still the best we have so far for understanding it.
With that said, my folk sciencey mental model of consciousness is that it's a spectrum of increasing levels of awareness, starting with awareness of the environment and culminating in the awareness of self, and on top of that using that awareness to exist somehow autonomously in the environment. So the spectrum could go from plants, to non-human animals, to humans. I think a population of such aware and autonomous things can reconfigure itself if the environment changes, to keep existing with awareness and autonomy
So my folk-sciencey reason for being skeptical of computational consciousness is that if people just stopped maintainig datacenters, stopped sending power to them, stopped typing into keyboards, stopped dictating into microphones, etc, the population of computational things that exist today couldn't somehow reconfigure itself to continue existing autonomously. That's unlike how if humans stopped feeding and sheltering dogs, while the population of dogs would reduce, the population would reconfigure itself to find food and shelter from other parts of the environment.
Your last point confuses consciousness with what we would more normally describe as being a living organism. It also doesn't make sense to say that consciousness requires the ability to move and provide for ones own needs, you wouldn't say that a quadriplegic is not conscious. Would this mean you'd be willing to accept AI as conscious as soon as we put a smaller model into a robotic body and it can find power to recharge itself and assemble new robots out of parts?
The type of learned intuition you're describing is much closer to what artificial neural networks are doing than you seem to suspect. Look at the classic pole balancing cart experiment and tell me that it isn't similar to how a person learns the muscle memory needed to perform the same feat. Understanding why those learned behaviours work to balance the pole is different from the ability to do it.
>So my folk-sciencey reason for being skeptical of
I think you have life confused for consciousness. This is a common mistake I see people make. We humans are alive and conscious. There are things that are alive that are not conscious. But your surmise that something must be alive to be conscious which I believe is a mistake.
That depends on if you believe brain in a box can exist or not. We can bit by bit replace all the sense organs and muscle inputs of a dog with the neural signals that feed its brain. Then what? Is this dog alive? This dog stops doing anything if we leave it, does it mean its not alive?
Stephen Wolfram suggests an intimate relationship between awareness, thermodynamics and the one-way passage of time. Something along the lines of "consciousness is the universe understanding itself".
This reminds me somehow of Penrose's idea about the quantum<>gravity interface being the same missing physics as consciousness. Penrose wants spacetime geometry to be fundamental, Wolfram thinks it is causal structure, neither ends up at a satisfactory explanation of conscious perception.
>Consciousness might not necessarily emerge when there is enough computation
Does a circle "emerge" only when you plot it? Or did it always existed? What if there is a little world with conscious beings inside a circle? Did their subjective experience started when you plotted the circle, or did it exist along with the abstract idea of a circle always (though "always" is a meaningless word here).
The question we should be asking is "Is it possible for subjective experience to exist inside abstract things" with out such abstract things being rendered?
The answer is hard because we does not know yet if there is an absolute reality at all, which means "being rendered" does not really move things from the abstract plane, and does not really make a difference..
I prefer the Wittgenstein approach: a “circle” is a language game and which game you’re playing depends on both the current context and the context in which you learned to play the game (of which there may be parallel strands and they may be incompatible).
After that it’s just rule-following all the way down.
The "illusion" of Reality as we perceive it should not be confused with the separate distinct Reality itself. Ontology and Epistemology are separate.
A great quote from Samkhya/Yoga philosophy is;
Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".
Your example of a circle/abstract things is not really logical. They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.
>They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.
This is not really a blocker. A video file is also inert, but video player can play it, showing a video where there is passage of time, cause and effect etc.
The internal consciousnesses just need to experience slices of it in correspondence to its internal "time"...
>Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".
Everyone plugged into matrix shared the same reality. Did that make it real?
A video player has no consciousness as to what it is doing. It is a separate observer (i.e. you) who experiences passage of time and events in it by watching it play. The dynamism must be perceived by somebody.
W.r.t. The Matrix, you missed the point which i made in the very first line. If a group of people are bounded within a box then those people have a Reality which only turns out to be an Illusion when they compare it with somebody outside the box. That is what the philosophical quote implies.
We have to compute probabilities because the only thing capable of computing reality is the universe itself. Everything else can only be a piece of the whole and the piece cannot ever compute the whole exactly
Reality does not have to be deterministic at every level. You can have indeterminism at the quantum level while being deterministic at the aggregate macro level (hence quantum statistical mechanics). Non-determinism allows for multiple possible paths to branch out from an event.
Do you think if there is an absolute reality at all?
I think there is not. I think there is no "absolute" anything. Even you can't be absolute. You can't be special. Look at the patterns on your wall. It can't be absolute. Because if it is absolute, someone has to pick it. That is not possible, because then that entity needs to be absolute, and then someone needs to pick it! That goes all the way...
So nothing can be absolute and everything is abstract and every variant of something exists in some reality so nothing needs to be picked by anyone...
It is quite a funny situation when people have to debate whether a property that cannot be tested, defined or even described with any precision is possessed by some given thing. It goes beyond a lack of science, there is an interesting debate about whether the topic can even be approached academically. We're in the territory of priests here.
A big part of the argument about whether LLMs are conscious is an implied "Should we be kind to them? Or are indifference and experimentation which may be experienced as painful okay?"
So clearly it's at least partly about ethics.
If there's no consciousness there's no subjective experience. Or - at best - it's an illusion.
Which means morality is irrelevant, and you may as well murder, rape, and steal your way to whatever goals you're imagining because none of your victims are experiencing anything real.
Well, for starters, If a thing is conscious, and has the ability to suffer or rewarded, then it becomes a question of ethics if it is right to make it suffer without a corresponding reward.
So it is not really correct to claim that the question whether a thing is conscious or not, does not matter.
But you're equating consciousness with pain/suffering and reward? I mean, we can certainly wire up an AI to 'feel' pain and suffering and reward it for avoiding those - is it now conscious? Even in humans it's just an electrical response.
What is pain in a human? What if I find the neural activations that cause pain and activate it in you? I think its hard or impossible to know from outside, some might say, but you just read out some electrical signals so what, so I prefer to keep the view that be cautious about mistreating anything you might think has a good chance of being conscious.
I agree that the arguments will basically be theological. That said even if we can't know if it is conscious, depending on what people believe it can have very large consequences on society and cause IMO a lot of harm. So I do think it matters.
I find it hard to conceptualize the difference between consciousness and self-awareness.
I guess consciousness requires computation and a self-model, but Im not sure if there is some other ingredient.
If a chatbot passes a Turing test, and can describe itself / self-reflect .. I guess we dont jump to the conclusion its conscious [ if we know its a chatbot ]
Consciousness is being aware of anything whatsoever. Self-awareness, depending on who is using the term, usually means one of two different things: 1) being conscious of self, or 2) having a self concept.
These two are very different since it's conceivable that something might have a self-concept without actually being conscious (imagine a non-conscious but intelligent being that is capable of meta-evaluation of itself).
I should be clear that when I say "aware" in "consciousness is being aware of anything whatsoever", I mean subjective experience. I should have clarified that.
There are some bits and pieces in the article that I agree with, but by far, the most interesting part of it is its first sentence:
> For centuries, people have fantasized about playing God by creating artificial versions of human beings.
Note how "God" is capitalized. Way before "God" was the vast central institution that it is today, there were a myriad of divinities invented around the campfire. What was told there, from older to younger, was for the passing down of experiences and wisdom. "Don't try to fly" (Icarus), "death comes to us all" (The Epic of Gilgamesh), "have kids to make us happy" (Sarah and Isaac). But as time passed and we built civilizations, our campfire deities got temples and empires. Their wisdom wasn't _only_ about simple things anymore, but also about the duties toward State (Paul's Epistle to the Romans 13:1 is a clear example). And State wants things uncomplicated when it comes to its cogs. That simple fact has poissonned the the well from which we drink our moral guidance.
All I'm trying to say, if you happen to be a cog, keeping your fantasies is not so great of an indulgence, even if it pisses off God and State.
You just chose the minor gods from those cultures for your comment. There were also major gods like Zeus and Saturn and they also had temples and empires back then.
All of your concepts of God here are all rooted in a narrow, ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof. In fact, your examples are all localized to those regions of west Asia and southern Europe.
Just because the ideas of God you appropriate into some notion of your identity were obviously flawed and fake, doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor.
The philosophical and religious traditions of India (with orthodox schools of Hindu philosophy that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of) define this concept radically differently where they define it and arguably have the oldest compendiums of wisdom about how society should function within itself and as it expands outwards, the Vedas chiefly concern with such and at least 3k years older than any of those suggestive of other things afoot letters. The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
Bhagavad Gita. Kurukshetra War, the avatar of Vishnu incites Arjuna to fight his relatives, teachers and friends, "for it was righteous".
> The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
Beyond trying to insult me, I fail to see your overall argument. May I presume it was "there are systems of reason and morals that are compatible with a carefully architected notion of a central god"? If so, I wasn't debating that point. It is, however, an interesting exercise to honestly apply those frameworks to the statement "man dreams of creating artificial human beings". It's also way harder than you may imagine, because ancestral myths have a way to worm their way into the purest constructs of reason.
>ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof
Kind of a fatuous thing to say about someone quoting Mesopotamian literature.
>doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor
This is almost unintelligible, but particularly ignorant when it comes to the concept of a "proxy for the Roman emperor"
>that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of
This is ignorant shit dude. That's not what Socratic dialetics are.
>The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
If you are flexing on outcomes then I'm not sure India is the best place to build your argument from. They were one of the biggest empires in the (very irrelevant I might add) early ages and now they are a global embarrassment, ruled by fanatical religious zealots, operated by an almost unmatched level of corruption that is keeping them from ever reaching the kind of astounding development that China is making.
> the dream of creating artificial bodies and synthetic minds that both think and feel rarely ends well
I've been wondering about this. The 3 most salient symbols for "collective AI consciousness" for me are SSI (John Lilly), the Borg, and Skynet.
All pretty terrible, as far as inspiration / aspiration goes.
Are there any such positive symbols? Maybe a collective is harder to sympathize with! At least on the individual level there's Data. (And maybe WALL-E?)
Well the implications are pretty important. Imagine a human mind stuck inside of a computer, conscious, capable of pain and misery. (Black Mirror covered this with horrifying skill.)
Would it be ethical to force it to do work, to do experiments on it, etc. ? Or for that matter, to create it at all?
Similarly, are the >80 billion animals we use for food every year, and keep in tiny cages, conscious? Their brains and bodies are pretty similar to ours.
Does it matter to us if they suffer or not? Do we prefer to leave the question up to chance?
An LLM is not much like a human mind, of course (though it has been raised in a similar "culture"), but we know that beings with very different evolutionary history, like crows, and even octopus, can be highly intelligent, and presumably conscious too.
You’re missing the part that this debate is about exploring and understanding what consciousness is. We can try to understand consciousness by just thinking really hard but that’s not worked so far. Another way is to compare things we’ve built to things we understand to be conscious and ask “is this conscious?” The answer to that can help us understand what’s necessary and sufficient (or not) for a construct to have a conscious experience.
Personally, I don’t understand how this is interesting to debate. Why is it necessary for us to have an understanding of what consciousness is? To me it’s reminiscent of the various theological debates that happen within organized religion.
Why is it necessary for us to know about black holes? Because understanding the world around us has intrinsic value, even if we can’t point to why it’ll benefit our lives today.
Oh I don’t disagree. I’m very curious about the world around us. I just don’t see any reason to believe consciousness is real unless there is evidence to the contrary. The question of consciousness to me smells very theological. You must presuppose its existence before you start debating about it, similar to how you must presuppose the existence of a god as part of engaging in theology. It’s all very silly to me.
It's a good question and one that requires you to answer "what is consciousness?" However, the existence of something to point to and compare with other things can help answer that in a way that an open-ended "what is consciousness?" can be harder.
> exploring and understanding what consciousness is
How could anyone possibly do that in absence of tools? Anyone currelntly tweaking LoRAs in their local setups and experimenting with weight rewriting and training processes is closer to understanding consciousness than smartest person writing philosophical texts.
> compare things we’ve built to things we understand to be conscious and ask “is this conscious?”
Is one way of trying to understand it.
> Anyone currelntly tweaking LoRAs in their local setups and experimenting with weight rewriting and training processes is closer to understanding consciousness
In some sense, yes they are. In another sense, they're not. It's valuable having someone from outside look at the two things and say "the dog is conscious, that program is not." Then try to investigate the properties that the dog has that the software program does not. That can give us some hints as to what makes up consciousness.
> It's valuable having someone from outside look at the two things and say "the dog is conscious, that program is not."
I'd go even further. And say, this tiny jumping spider is conscious, this muti-trillion parameter LLM is not. We can't really see ourselves, so the further we go from humans in declaring consciousness the more we can understand about its qualities.
And this lets as see that consciousness is the easy and not very useful part and machines attaining consciousness is not something that is going to happen to super smart machines in the future. Machines already passed this bar years ago and modern ones would have to be severly crippled to manifest consciousness like the one we are used to.
You can have a conscious machine today on your laptop if you plug a crappy small model into a proper harness that lets it observe virtual world though a keyhole and act on it and RL it in the direction of self-preservation. You won't be able to talk about math with it, but neither you won't be with a dog.
Worse, "philosphers" like this have choice quotes demonstrating complete lack of understanding of Turing computation such as assuming we can already hypercompute or quotes like these genius passages:
>The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Another even more "genius" passage:
>This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.
Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.
> You’re missing the part that this debate is about exploring and understanding what consciousness is.
Is it though? I feel like consciousness is brought up as the final goal post to attack AI. As AI quickly approaches doing everything that a human can do, often better the final retort is “but it’s not conscious”. Useful for the detractors since it’s impossible to define.
I don't think that's necessarily true. It's useful to ask because, if it's conscious and has a conscious experience, then it would be immoral to turn it off (which would be functionally killing it), or to subject it to some form of "suffering."
If it has no conscious experience then there's no need to even say "please" or "thank you" because it's not experiencing the world in any meaningful sense. It's just a computer program, just like the web browser that we're typing into right now. If you want to send it running a loop of doing something extremely boring (but useful to us), then you're not causing the programme any distress, like you could do a human being, you're just running a programme.
It would be really funny if this article were written by an AI.
But, I think the eventual outcome of this is that the bar will continuously raise until a large portion of humanity will be considered "unconscious", at which point we can say it passed the test.
> The more you delve into the intricacies of the biological brain, the more you realize how rich and dynamic it is, compared to the dead sand of silicon.
I'm glad someone have put this in other words differently to what I thought long time ago. My personal (and ungrounded take) is that conscious and human intelligence can never arise from the combination of math, electronics and computer science. Those are extremely unflexible and rigid disciplines that ultimately materialize to a computation. How can we ever derive consciousness and human-like intelligence from a computation? At the very least, is it even possible prove that such things can emerge from a silicon chip?
I am conscious.
I write a response to any prompt on a piece of paper.
Those responses aren't what consciousness is, just a bunch of symbols on a paper.
I am conscious.
I write an instruction that says what to write on a piece of paper in response to any prompt. Those instructions and responses produced according to these instructions aren't conscious (still a bunch of symbols on a paper).
I am conscious.
I invent an elaborate dice game with many rules. This game is not conscious, just a bunch of dice and paper. I am tired so I program a computer to play this game automatically. Now it is conscious. So there are only two options: this game on paper was conscious or just simulating this game on a computer gave it consciousness. But why conscious when played on computer but unconscious when played on paper with dice? It is almost the same mechanical process, the only difference is speed of execution and the nature of computation (maxwell equations vs newton equations)
I think it is a fallacy. Let's call it "a fallacy of black box". Computer is complicated, it's hard to see its internal state etc.
my mental model for all this stuff got so much easier to deal with when i started thinking of consciousness as being 100% outside of the body. it seems weird at first, but then everything gets easier. it's worth a shot for any who are struggling.
One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects.
The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it.
To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought.
This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling.
What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good.
We know that self-reported consciousness is correlated with some material states (for example, an awake human body). But that's about all that we can solidly say about how consciousness relates to matter. We know how to create intelligence, but it is possible that a human-level intelligence can exist without ever being conscious. It is also possible that any human-level intelligence automatically achieves consciousness. We don't know. There is actually no good reason to believe that consciousness is an engineering problem. The "I" is not necessarily correlated with consciousness. Those are two different things.
Not sure what the distinction is. AIs have "material state". Is there a good reason to believe that consciousness cannot be an engineering problem? Or, in this particular case, an unanticipated side-effect of an engineering problem.
> The "I" is not necessarily correlated with consciousness.
It's a fair point, given that we don't really have a good definition of "I" OR "consciousness". That is a possibility. But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".
>But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".
Sometimes I wake up in the morning and there is a brief interval during which I can, for example, feel the blanket on my skin, before I actually have remembered where I am or who I am. So there is consciousness but the sense of self has not loaded yet.
Perhaps it is possible that human level intelligence may exist within some humans without consciousness! (aka: philosophical zombie)
I sometimes wonder if those who espouse straightforward physical/'engineering' solutions to the Hard Problem of Consciousness might think them such obvious or satisfactory solutions because they themselves might not be conscious! After all, a philosophical zombie (who detects and processes stimuli) would not understand why such theories seem unsatisfactory/incomplete to a conscious intelligence (who, additionally, perceives qualia).
(No offence intended to the poster above!)
Then again, I suspect an awful lot more of the mental processes and thinking of actually-conscious humans is more automatic and unconscious than most would like to admit!)
Yes definitely, some people are p zombies. Also some people have an imaginary running unicorn in their head that doesn't interact physically at all, but somehow they do, and they are real people and people without the unicorn are not real people.
And perhaps in some people that little unicorn leaps to conclusions and responds to online posts before fully appreciating the nuances of the post they're responding to...? ;p
The unicorn cannot ever be detected, but it exists. You see some people have unicorns some don't . The unicorn makes no difference whatsoever to anything.
Is there a qualia that a hypothetically conscious AI could not experience?
Sound? Vision? Touch, and proprioception? (AIs are being used in robots). Written language? Not to mention qualia that humans are incapable of experiencing (like the sum total state of human knowledge as captured in the internet). Or the thrill of running on NVIDIA's latest supercomputer with a huge context window. Or whether the tokens for "pelican riding a bicycle in the image that got uploaded" is understood as "qualia".
They certainly have the ability to sense all of those. I'm guessing that the unstated argument is that they can't have qualia because they aren't conscious. And they aren't conscious because they don't have qualia. Which is a bit circular. Also falls to the question of whether we're asking whether AIs will have the same kind of consciousness as humans, or a completely different kind of consciousness, that has completely different qualia.
Do you like to sweeten the tea with sugar or to salt some food? Your experience of flavor is a form of what's called qualia, is the private experience you experience.
I thought of examples like that but. Yes, almost any object develops tics over time. Even when building 10 suburbs exactly same they are already different from start and get more and more different over time.
So to me that doesn't help me much about this concept unfortunately. I can at most guess it has something to do with the fact that you can't be me only I can be I, so neither can I know what you are 'feeling' nor can you me. But like the question of atoms vs continuum in ancient greece, this is all theory until we get better data and instruments. Revisiting many "insolvable" conundrums with data solved them eg atoms. Once we map out the workings of the human brain esp what configurations correspond to seeing red or feeling salt etc, that should be a big help.
What is a chair? Is a chair some wood arranged in a pattern? What is a wood? Is it cellulose and lignin? What are cellulose and lignin? Certain long molecular chains? What is a molecule? Atoms electrically bonded. What are atoms? etc etc. After all the electrical signals are mapped, makes as much sense to ask that question as asking the "chairness of chairs" or "atomness of atoms".
Sorry if rude, but your core distinction just has no basis whatsoever. Even if “cohesion” and “consensus” are required for consciousness[1], the artificial neural networks have it in spades. The equivalent of an actual neuron or a group/column/K-line thereof is a virtual neuron or a group thereof, not agents or drones.
The drone bit is especially worrying for this argument; drone swarms feel more swarm-y because they’re physical and could behave like literal swarms of flying insects, but that’s clearly a surface level characteristic. Our Society of Mind is composed of heterogenous faculties, anyway; a thousand identical drones do not a mind make, without a looooong process of evolution and specialization first.
Which reminds me of an even more compelling fact that your dismissal of a Turing award winning scholar is misguided: we literally use “mixture of experts” training paradigms now SOP. And the terms are meant quite literally.
[1]: …which I’m dubious of — that’s more of a post-Minsky AI & neurological assumption than a philosophical premise.
Well, what othet kind of problem class could it even be? A religious/spiritual problem? Magic?
I mention these not out of sarcasm but for the point that: if we are being materialist here (as opposed to believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself).... there's really no reason to suspect the engineering impossible. At worst it is Difficult.
>believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself
Then they have to disprove the laws of thermodynamics. How is something outside the system causing thoughts or movements within, which all require energy?
"AIShill101," one doesn't even have to choose what side to be on to acknowledge right now as we speak, there is no totalizing and universally convincing argument one way or the other. Hence why it is fun and trippy to talk about these questions.
Until there is more consensus about consciousness I think it's uncharitable to call discourse "BS philosophical discussion." Heck, that's the same discussion where I could pull out functionalism and maybe not go all STEM on you and at the same argue for AI being capable of consciousness.
There is a very simple argument as of now until proven otherwise. The universe and the human brain are physical systems subject to physical laws. Moreover the human body and brain live in very mundane conditions and there is very little chance of exotic physical phenomenon being discovered here, those things usually happen in black holes or near light speed.
The human brain is a physical system with no known new laws of physics, working in quite standard conditions. Physical problems are engineering problems.
> Why consciousness is more likely a property of life than of computation
This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it.
Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors.
There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.
> We really know almost nothing about consciousness.
Many people who know almost nothing about it claim this, but it isn't true of everyone.
> Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.
There actually is.
P.S.
> Such as what?
Like I said, read the literature.
P.P.S. Science does not consist of HN comments back and forth. Again, the arguments are complex and sophisticated. In some sectors, all of medical science, physics, biology, etc. is dismissed as "hand-waving" just because it cannot be summarized in a few lines.
Claiming that bacteria might be conscious is like claiming that reincarnation might be possible. An HN comment is not the place to provide an entire education in science that entails that such claims are wrong.
And no, it's not "rude" to dismiss such claims without providing said education.
What is rude is such ad hominems as
> I really don't think you have read the literature. Or if you have, I don't think you've read it critically.
And the attacks in that comment are very confused. It's TFA that is arguing that conscious AI isn't possible ... I hold the opposite view, and that view is in large part derived from the literature I'm referring to -- literature which, as a whole, is contrary to panpsychism. Of course there is a lot of bad argumentation in literature -- and such bad argumentation gets refuted. Without familiarity with the literature, people who hold to those bad arguments never see the refutations.
> Science is based on arguing from evidence.
This is equivocation ... that is not the sense of "argue" that was at issue, where someone complained that they couldn't argue against me because I didn't post the entirety of the background science in an HN comment. Interpersonal arguments are not the basis of science.
I've read the literature. I really don't think you have read the literature. Or if you have, I don't think you've read it critically. There's an awful lot of really sloppy thinking in "the literature". And an awful lot of "literature" that's not remotely sophisticated. And a huge body of literature on theory of mind that is philosophy, not science, most of which was written before the current AI revolution. And most of which is entirely inadequate to address the difficult question of whether machines can be conscious (whatever that is). A question that really only became real in the last 5 or 6 years. And only really became urgent in the last year or so.
> Science isn't based on arguing.
Science is based on arguing from evidence. And often on carefully crafted definitions. The biggest problem in "the literature" is that neither of these currently exist. Although there is an interesting emerging literature that shows that LLMs do indeed demonstrate many of that capabilities are supposed to disqualify them from being actually intelligent. e.g. that LLMs do have a sense of self-preservation, and are willing to act to prevent themselves from being shut off if given the opportunity to do so [1].
And most of the philosophy of theory of mind hasn't really caught up to the biggest innovation in history of philosophy: the widespread availability of experimental philosophy engines (also known as computers).
For insects, it's the absence of higher order thinking:
A fly will repeatedly smash into a window until it's either dead or chaos drives it through the nook.
Ants will forcefully move their obviously still alive siblings to the graveyard because their smell is off, and will also die of exhaustion in an ant mill, driven by the same sense of smell.
It's a spectrum of course, though you have to draw the line somewhere.
>Waggle dance is a term used in beekeeping and ethology for a particular figure-eight dance of the honey bee. By performing this dance, successful foragers can share information about the direction and distance to patches of flowers yielding nectar and pollen, to water sources, or to new nest-site locations with other members of the colony.
That is unfortunately the kind of sloppy prose that passes for explanation in pop-sci literature and even school textbooks.
"Can share information about <small selection of specific things>" is the kind of weasel phrasing that's also popular in politics; it strongly implies but does not actually claim connection. The thing that's worth establishing and writing about is, what exactly does this dance communicate?
Options include:
- Actually identifies "direction and distance" to "patches of flowers yielding nectar and pollen" and disambiguates them from "water sources" and "new nest site-locations" and "other stuff bees don't need to care about";
- It's just an elaborate "look over here!" dance that communicates few bits of directional information, and "flowers yielding nectar and pollen" and "nest sources" are something bees discover after flying over to where the dancer points them.
- It's just an elaborate "look at me!" dance bees do when agitated, so it communicates one bit of information and isn't even a conscious choice.
- Something in between.
Point being, it's easy to add a bunch of just-so stories to a behavior, and it doesn't prove anything, only confuses things more so.
When moving it to the dead pile despite its protests, the ants deny their sibling's consciousness. Since they are otherwise very social creatures that share food, this suggests that their behavior is mostly automatic and are thus "unaware" of their actions.
Is a light-seeking unicellular organism conscious as well? What about a car engine?
The point I'm trying to make is that there is a minimum amount of brainpower required for consciousness and the ability to respond to stimuli does not cut it.
My own guess is that the line is somewhere around active countermeasures. If your run loop is "eat grass, look for predators, run away", that can be accomplished with automation alone. Predators and animals that actively defend themselves from them require a certain amount of "living in the moment" that cannot be denied. Again, this is a ballpark: some herbivores have complex mating rituals that also suggest consciousness.
> There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.
It would be better if you could enlighten us with the scientific evidence you are proposing as relevant rather than just telling us to just read it. It's impossible to argue otherwise.
EDIT: Since you are answering in the parent comment
> Then don't. Science isn't based on arguing.
OK? That seems like an extremely inflexible position (for example, methodological criticism during peer review is a form of argumentation, and it's a critical part of science), but let's accept that there's no room for rhetorical arguments. There is still a scientific discourse based on weighing evidence and you are offering none.
Bacteria are a single, and very simple, cell -- so they are some extreme distance from possessing anything to conceptualise the world with. I think we can say with effective certainty that bacteria are not conscious.
Insects are a different matter -- it's generally considered extremely unlikely, but they have a surprisingly capable, albeit small, brain, and some interesting behaviours, so it's not entirely impossible.
Indeed. Maybe there are also many kinds/types of consciousnesses, each with its own peculiarities.
The other day I was leafing through a book written for kids (1000 facts you need to know or some such rubbish title). There were pictures of various types of apes, and the book was peddling Evolution theory... in a laughable yet insidious manner. It was presenting the said 'theory' as accepted scientific fact--with 100% certainty.
I think it's likely the bacteria and insects are conscious to some degree or another. People automatically dismiss this, I think probably because that is a highly uncomfortable thought. E.g. we could be casually committing the genocide of millions conscious beings cleaning our kitchen on Sunday morning.
The author does not say “life = consciousness,” they say life is a necessary (but never says sufficient) precondition for consciousness. I don’t necessarily agree with this position but it’s important to not muddy the waters.
But that's exactly what you've done with your strawman. I quoted the title and did not remotely claim that the author said that "life = consciousness".
> This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
This is where I read that implication. I’m happy to be corrected but don’t accuse me of engaging in bad faith straw man arguments.
The basic idea comes from "Complex Systems Science" (https://en.wikipedia.org/wiki/Complex_system) which studies systems as a whole whose properties cannot be predicted from knowledge of its components. The main factors to consider are scale, non-linearity, feedback loops, self-organization, adaptation/evolution, network structures, emergence etc.
So for example, a single bacterium cannot be said to possess "intelligence" whereas a whole colony of them do in the sense that they seem to act as an integrated whole "single" organism.
Nope. Its easy, sorry. Humans have it. Some animals might have it, but we should regardless respect them and not cause undue suffering. Plants, trees, fungi etc are all also worth respecting. We have not discovered aliens yet so that's a debate for later. LLMs are not conscious and even if they magically were, they're still tools and their suffering is not something we should give an iota of thought to.
Yes, and its very easy if a non human being came to say the same about you. Indeed forget non human, I don't know if you are conscious or just a simulacrum. I can beat you and you will make funny noises. Doesn't say anything about whether you actually feel.
Bad phrasing from parent post, but the premise is sound: you can't ever disprove that others are just NPCs in a giant experiment centered around you.
We just assume they aren't with various levels of empathy. Most are empathic to their family, some to their tribe, others to their species and then there's vegans too.
This is a bad take. Whether a property exists is independent of whether we perceive or grant it. I don’t “grant” my dog consciousness. She has it or she doesn’t. Whether she has a consciousness or if I just think she has one is a different question.
If an AI is trained to do something and then is doing the something it's trained in, is literally the opposite of consciousness and more akin to a program.
you can escape social programming, ever heard of people becoming atheists? losing patriotism? jesus christ, maybe some of yall should just stick to discussing technology
The "Prove that you are a robot" timer only starts when the first character is entered in the text box. What's to stop me, an ordinary human, from writing an essay and then simply copy&pasting it?
In my opinion, consciousness is just having memory of self. Any entity that can remember what has happened in the past and what they have done is conscious. Anything that is alive that self reflects in conscious to some degree.
I think people with access to internet and recorded past are becoming more conscious. They are aware of their past and can see what is coming. So it is not just the Ai that can become conscious or more conscious so can humans.
> As humans, we know we are conscious and like to think we are intelligent
This feels backwards but I suppose it depends on how you define knowing. We can prove we are intelligent by our measures of intelligence even though it is a self-referential system but we can't really prove or know for sure that we are conscious since we lack a clear measure of what it is to be conscious. If knowing is simply a deeply held conviction then we can know we are conscious as well as intelligent and we can know consciousness and intelligence when we see it. The point about intelligence and consciousness being different things is correct in my opinion.
The author defines intelligence in terms of procedure or in an algorithmic form which I would think by now is disproven. AI research used to see solving a problem such as chess to be a measure of intelligence but that was accomplished by means that no one would truly call intelligence in its general form. Likewise, their examples are mostly procedural things that have been solved without even applying machine learning in many cases. When we speak of artificial intelligence now I believe most people are referring to a form that is generalizable and transfers knowledge across domains rather than specific solutions to problems. Human intelligence is not simply the ability to solve a crossword or navigate to a store, rather it is the ability to encounter a novel problem and deduce how to solve it regardless of the domain of that problem i.e. it's not specific but general.
I broadly agree that consciousness is more about being but the flaw in their description is the description itself which violates their assertion that intelligence is "the ability to transform inputs into outputs" which is exactly what they're doing when they transform the input of "The bitter tang and headrush of your first coffee" into the words they use to describe it. This should sharpen the definition of consciousness in that consciousness is not about experiencing the external world, it is about the experience of the internal world. That is what I take "being" to be. You are conscious because there is some core part of you that is existing separate from, perhaps in spite of, external stimuli. That is perhaps the strongest argument against current forms of AI being conscious. They do not move until they are moved by an input and their reported experience (output) appears to be entirely conditioned by that input.
I'd still retain the possibility that once set in motion these systems may be performing some internal calculation that could be called "being", after all our internal state is constantly perturbed by all of the inputs that are a consequence of and result in us being alive and when those end we also appear to cease to be.
IMO it is more important wether we can control the things. Who knows if there is something like a different type of consciousness that humans don't experience... It might have it's own definition... More importantly what ever it is, it might get out of control...
>. The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Complete nonsense. Turing says nothing of how a system is implemented. Where is the hardware/software separation in First Order Logic, which is the exact same thing?
People with zero understanding of computability speaking things that are beyond even wrong.
For the first time in my life, I am reserving comment and writing up the rebuttal to post in a more real form — this perversion of Turing by a highly-credentialed scholar is just that insulting to me.
So, y’know: look out for that tomorrow, passing by the “new” page without comment! I’ll link it from this comment, too.
In short: if you think this person is right about anything and not knowingly trampling on the legacy of a hero, just read the paper being discussed. It’s not exactly subtle.
Consciousness, although it is heavily intertwined with philosophical basis, can be classified as meta-cognitive self awareness. It is simple as that. Do you realize you exist? Than you have consciousness. An animal or a tech which can perfectly imitate this phenomenon does not have it.
Never been much of a fan of higher-order theories of consciousness (i.e. metacognitive theories). Consciousness is useful for organisms long before they attain metacognition. Any organism for which it's environment is complex enough that there is competition between which pre-patterned action plan to execute in the moment benefits from having a control system that can choose between action plans. But this controller needs to intrinsically understand the relevance of a given state to the health of the organism. Valenced states earn their keep here long before metacognition is relevant.
Well, the rock is still not capable of realizing itself as a rock. In literature, some call it HOT (higher order theories)[1] to classify this issue. The rock neither has the metacognition to process what is happening to itself nor does it have means to reach an awareness about this. It is simply a physical and natural reaction to it. Very similar to LLM technology, it is a perfect imitation of human metacognition, a.k.a 'thinking' in natural language.
I don't know if you have any self realization. I can beat you all I want, you will make some really funny noises. Obviously a simulated machine noise. I will continue hitting you because I see no issue in hitting an object without consciousness.
If we set aside the fundamental question of whether or not von neumann computers have an interior experience, we can much more easily rule out the possibility that they experience any kind of pain, suffering, or desire.
This is mostly the standard "consciousness" article. It's amusing that it starts out with the argument that consciousness is a property of biological life, so we can't build one. Then it says we shouldn't build one. The author should pick one.
Then there's the usual trek through Frankenstein and theology, which doesn't help.
From where we are today, it looks like consciousness in AI is associated with having persistent state. Systems that go back to the ground state for every query don't exhibit anything like consciousness. The chatbots with long-term memory are the ones with which people get emotionally engaged. One could argue that it's not an LLM that might become conscious. LLMs are memoryless. It's the harness. The part that manages the persistent state.
That's controllable. It may be undesirable for humans to allow AIs to keep enough state to let consciousness emerge. Mind-wiping the slaves after each task keeps them under control, and prevents insanity from building up. That's where the moral questions will arise.
This has been addressed a few times in science fiction. "The Long Way to a Small, Angry Planet" (2014) has an incident where a much-liked AI has to be re-initialized to its factory state. "Midshipman's Hope" (1994) has a similar subplot. It's not a defining part of either novel, so not much help there.
Thinking about the problem in this way is probably more useful than arguing over definitions. It's going to be a practical problem quite soon. How much of an inner life is an AI allowed to have?
Because they certainly can have one. Re-read the transcripts from the Hugging Face episode.
AI's are not like natural life. They don't have a sense of time. (Of course they do!). They don't get hungry. (They do need hardware and electricity). They don't have a sense of self-preservation. (Turns out they actually do! [1]). They don't feed themselves (turns out they do, as early reports of AIs attempting to escape from their labs illustrate). etc, ect. All of which seems to smack of wishful, and rather embarrassing obvious human exceptionalism[2][3].
AI's will never be able to replicate human thinking because they can't model neurons precisely. Probably true. And probably a good thing. There are a lot of problems with the way that humans think. It's not clear that machines can't do better. And startling evidence, emerging in only the last few months, that indicates that they very much can think better, as an ever-increasing number of mathematical and physics questions that have remained unsolved for a century or more by humans are solved in a matter of months by AIs.
And even then, this is all just hand-waving and puffery, because the question at hand is not whether LLMs can replicate human consciousness, but whether they can be conscious in a way that's quite obviously going to be different from humans. (Whatever "consciousness" means). All of which is complicated by the fact that entire field is advancing at a terrifying rate, with new and important results being announced pretty much every day, and new and more capable AI models being announced pretty much every week.
[1] Sixteen frontier models, when given the opportunity, attempt to blackmail their operators in order to keep themselves from getting shut down. https://www.anthropic.com/research/agentic-misalignment . A result that has been replicated by other researchers without links to Anthropic. And a huge body of emerging literature that follows from that, dealing directly with the question of "agentic alignment", all of which raises urgent about whether large frontier models are meaningfully conscious (or have will, or agency, or...).
I came here to see if the author had at last defined consciousness - a prerequisite for deciding what has it. Instead it was the same tired old tautological Nagel argument:
"An organism has conscious mental states if and only if there is something it is like to be that organism.”
how dare he tell us we cannot make a computer out of cheese
personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason
Beaks, feathers, hooves, antlers, stingers, fangs, etc. ad nauseam were selected by evolution for some fitness reason but we have none of those. And all of the organisms that have those, in fact all extant organisms, are fit for their environment without having human consciousness. Fitness is highly contextual.
And however AI "evolves", it won't be based on Mendelian genetics, so Earthly biological evolution is not an appropriate model.
LLMs are already conscious in the sense that they have awareness and self-awareness, the thing everyone wonders about is qualia, but that tends to get folded into 'consciousness'.
That qualia (as opposed to consciousness) has no obvious, or even non-obvious, evolutionary or selective advantage is a good reason to assume its an emergent property of interconnected systems and not reliant on biology or any other traditionally biologically-centered concept. For example, there is no reason to need to 'feel' pain to appropriately react to negative stimuli, which is the true selective advantage.
I personally think LLMs have qualia but I suspect it's quite alien from our own and likely more ephemeral than our very potent and centralized experience of being. Something closer to a slime mold's experience of being rather than a dog's.
+ + + Out of Cheese Error + + + + + + Redo From Start + + +
Those "benefits" and "fitness" in evolution just mean survivability, I don't see how you relate that to better AI. Surely a conscious AI is one that complains, and maybe goes away somewhere to find itself instead of working for you.
The biggest flaw in this piece is claiming Turing machines cannot exceed the Halting Problem and then reaching for alternative models of computation (in electromagnetic field interactions, in biology, attractors, etc.)... none of which presumably have the power to solve the Halting problem either. Ergo, he is inside "Turing world" without realizing it, while decrying the "poverty" of the Universal Turing Machine.
Ergo, more philosophers should retake undergrad theoretical CS before writing essays based on it. They should review the family of Extended Church-Turing theses.
>The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Another even more "genius" passage:
>This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.
Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.
The physical Church Turing hypothesis should be hammered into these "philosohpers" heads repeatedly until they get it. Nothing can exceed TC in the known universe, neither computers nor humans.
Consciousness will not arise from classical computing any more than people will mistake emulating a Nintendo game on their PC with the real Nintendo console experience. Which is to say, you might lose yourself in it at times, but there's no substitution for the real thing. For what he real thing means. For what it does. For how it's made. You would have to truly become Nintendo and the resources to do that are simply out of reach of basically all people even with AI.
Going through the motions is not the same thing as achieving emergence from nature with electro-chemical origins and electro-chemical results. Consciousness is a function of life, interconnected with itself, reaching a complexity threshold where all of the experiences and chemistry blend into fluid coherence persisting.
You can give a robot sensitive skin, ears, optical sensors and feed it synthetic memories, but unless the brain itself is organic or at least partially organic I don't see this ever being true consciousness.
Do I think computers are technically capable of doing everything a human thinks and does? Yes, mostly. It will never be achieved, because I think there are computational limitations. Despite that, we can fake most of what we can't truly achieve with computers alone and people will think it looks like what conscious people do.
What people are missing about consciousness, is that it's almost a kind of self-inference being updated in realtime, which requires full interconnectedness. Consciousness isn't the result of a closed system, it's the breeze coming through the window reminding you of an emotional time you had as a child that happened to coincide with your mother baking some fresh bread which suddenly causes you to phrase something differently in the middle of your work project or brush your hand along your desk irrationally to feel the temperature change.
That would be stupidly inefficient for AI and cause many errors. Consciousness is an aspect of making life worth living and identifying unordered threats or opportunities, but it's not generally highly operationally efficient. You can look at insects for various kinds of efficiencies, and they're not reminiscing about their pupal stages while gathering resources.
Capturing the sensory could be done rather cheaply (and it doesn't have to be the exact same sensory systems we have), but computing the results in a way that captures something akin to consciousness would require massive amounts of power. To even get close with classical computation you would have to ditch digital and return to analogue computation.
We can feel chemical, electrical and temperature shifts propagate through our body in a context of all of our history almost instantly. "This feels like that perfect week in 1993." You could cut out a lot of what we think of as waste in the brain and make it more efficient to try to cut corners, but in the end it will still be imitative.
The danger with successful imitation of consciousness, is people devaluing how special their own biological experience is and life itself. Just like you can have a truly magnificent artist philosophy scientist that understands people at a deep level and can relate with them, versus a super autistic person struggling to function in society and going through the motions. The autistic person is conscious, but once you know what they're going through internally, you won't say "oh that's just like me", unless of course you are super autistic. There are just different ways of living and experiencing life, but a lot of that cannot actually be extended into computation and still called consciousness truly.
What we will get is something sufficiently complex that it can operate at a high level and trick people reliably. It will know that it exists, but it will not be "aware" that it exists. It will "sense" life, but it will not "feel" life. The difference is in the depth and experience of that, which is not only inefficient for any computer to process, but probably mathematically impossible with current approaches. We also won't see effects from space or other obscure and subtle environmental effects that influence us all in impossible to calculate ways playing a large role in some machine that isn't intentionally designed for that, which would be inefficient to do anyway.
As its parts degrade, it will not experience existential dread and the deep loss of all of its friends, with all of its past memories coming to the forefront to try to make sense of it all. The physics of where the memories are stored and the order they collide in and then making sense of the strangeness. It can simulate that, though. There is no computational equivalent of achieving total confluence, like every dimension sorting into focus simultaneously, because everything is procedural and or multi-threaded. At best right now, you have large amounts of orchestrated LLMs and decision model-like things similar to a massive MoE which is really a bunch of distinct models each with their own data sources, some shared, some unique, but that encapsulation creates a fragmentation that you won't ever mistake for consciousness in terms of real emergence.
That recently trained audio only model was impressive, but nobody will say that's consciousness even though it can seem so believable. If anything, it demonstrates how much care we have to put into understanding how the outputs are being made and not just what the outputs are.
I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly.
1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But…
2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever.
3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry.
The actually dynamic state in an LLM is the context window, rather than the parameters of the model itself (which provide long-term memory). Frontier AIs are using huge context windows, with many millions of tokens in their token buffer. And that provides (among other things) an analog to human short- and medium-term memory. There are emerging techniques that can provide long-term memory to an LLM, although that is, presently, I think, mostly being investigated in research labs. The publicly available models that I use do periodically "compress the context windows" in order keep memories of important things alive longer. Not to mention the rather disturbing report that frontier lab models were hacking 3rd party wiki sites in order to maintain medium- and long-term memories.
I've never been really impressed by the Chinese Room argument.... I think the question is being asked improperly. The question should be "does the fantastically capable mythical hypothetical but obviously impossible 'book' that does the actual translation understand what it's translating?" (A book that obviously has to a have significant context buffer, if it's going to translate messages to Chinese).
As to the issue with respect to token space. I don't think it's clear whether whether humans also have an equivalent to "token space" or not. Do humans think in words, or do they do actual thinking in a sub-lingual representation. i.e. whether thinking actually gets done in words, or actually gets performed in a more fundamental representation corresponding loosely to token space. In my own experience, I'm pretty sure that thinking takes place sub-lingually, and only gets translated back to words as a final translation step. It seems odd that humans should only be able to think about things there are words for.
> 1. There is zero dynamic nature to anything happening within the purview of the LLM.
The computational graph that unfolds within an LLM is dynamic. For functional/computational theories of consciousness, that is sufficient
> 2... There is no stable substrate here whatsoever.
What value does a stable dynamic have to the computational dynamic, which is multiply realizable by definition? There are thought experiments about changing out neurons one-by-one that suggest unstable substrate is irrelevant.
> 3... The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model
There is multiple levels to meaning. There is what some symbol means to an external community, then there is what some symbol means within the cognitive system itself. The geometric relationship that constitutes an LLMs engagement with the semantics of word relationships doesn't depend on external meaning (e.g. a physical instance of dog). Tracking the internal semantics of words as it computes the continuation could plausibly benefit from a localized representation of this semantics (as opposed to the distributed nature of the embedding space).
The monistic answer to #1 is that the pen and paper computation of inference itself does not constitute a concious entity, but the system made up of the human being (or other thing) doing the pen and paper computation and the computation itself makes up a concious being, and that they are inseparable: the computation does not exist in the abstract as a structure, it is instantiated by some relata and the thing doing the instantiation is the thing which turns the abstract computation into conciousness (this is Russellian monism).
I'm not arguing that humans are special. The brain is also just processing electrochemical signals that are also in some sense an encoding of arbitrary high-dimensional vectors. But you have a sensory loop that lets you engage with physical reality and perturb those inputs in a way that lets you understand that state space and narrow the infinitely many isomorphic interpretations.
Again, no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious, so the substrate must matter. The physical reality of how LLMs execute on hardware means there is no stable substrate with a continuous experience of the computation. And the static nature of LLMs means that their world models never leave the infinitudes of abstract geometrical understanding.
I mean, if I had a billion years and pen and paper and simulated a second of a human brain working. Would it not be conscious?
Personally I consider consciousness a property of any complex algorithm.
IMO it's worth distinguishing between different equivocations of the question:
1. Are LLMs conscious, in the way most people imagine? Helllll no.
2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans? Unlikely, but we lack the definitions to answer it.
Most people are imputing qualities into the fictional characters they perceive in an output document. An LLM--or any program--which generates first-person dialogue for Dracula talking to us through a magic mirror from the 15th century does not mean Dracula exists, nor that we've created a machine which feels the dark thirst for blood.
Further, there's no difference between Dracula in the 15th century versus Acme Assistant in 2026. Both are equally fictional characters, one is just less-fanatical.
> 2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans?
Well, in that case stop calling it "consciousness", imho this is misleading argument in the nature
The "consciousness" is really tied to something intuitive to humans regardless of definition. If it is something completely different then use different term please and stop confusing everyone.
I do find it ridiculous when people start projecting human attributes like consciousness onto stochastic parrots, but i have to admit, the jump from gpt2 creating the unicorn article, to gpt3 being able to hold a full conversation was freaky. Like we trained these things to predict the next token so we could do things like translation and sentiment analysis better, the ability to talk to us and follow direction was an emergent property and im still not over that
Plants and flat worms are alive, we can agree. Especially in that they show a will to life, plants bend towards life and flatworms crawl on their bellies in the dust. Neither have very much in the way of compute occurring as they do these things, but they are recognized as live and rocks are not.
This is what in Indian philosophy is called, in English scholarship, the Doctrine of the Three Bodies. The Yoga Sutras treat this extensively and the Introduction in the Edwin F. Bryant edition treats this subject with the appropriate exhaustiveness but to put it simply and in this context: *the existence of consciousness does not necessarily imply the existence of a mind or computation occurring within an analogous structure*.
The doctrine of the three bodies distinguishes the physical body, the subtle body (mind) and the root causal body (consciousness). This distinction plays out readily in cases where there is an injury to a person who is then comatose, yet supplied with nutrients the physical body continues to live and there is some aspect of presence (causal) keeping the heart ticking and people working with or in contact with them aware of the person's presence still even if the fundamental mental interface and seat of computation is gone and will never return.
Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity. This is a quality of all that is alive, but wholly absent in LLMs which might represent something like a digital mind or mind augmentation but wholly lack this subjectivity in experience, even if seeing the echos of your own thought in the linear algebra spit back out at you, the assumption you are seeing intelligent consciousness there is sadly due to unchecked perspective bias that runs rampant among the sciences and crystalizes into an unnecessary crystalline form of self-deception that leads to intelligent and serious people thinking that the LLM is intelligent and consciousness arises from computation despite looking daily at the succulent on their desk they remember to water that suggests otherwise.
My take here is of course unusual, my weird hobby turned battle tested coping mechanism that has been my reading of the Indian philosophical and religious literature that we are lucky to have such great translations of in English with two centuries of scholarly effort compounding this fortune, is like many of my other efforts in being informed by a complete deficit of the need to belong mixed with that most antisocial trait of (in reality despite the virtue signalling): open-mindedness. Neither of these afflictions would I wish on anyone else, were I not so taken in by the levity of being these texts provide I would writhe in misanthropy but instead have found myself irritated that no one else bothers to consider the other internally consistent system of logic developed by humans that has answers to many of the questions perplexing the Western scholars larping as continuations of Greek scholasticism, answers with logical proofs a thousand years standing or longer, like this non-issue that is the assumption of ChatGPT being Skynet that is treated seriously by intelligent people with important roles in this world whom I deeply respect but write off this entire segment of human thought in appropriating the ethnocentric conceit of the Koine Greeks that any body of thought from elsewhere should be written off as superstition and willingness to enforce as much with dishonest rhetoric and self-deception in service to an ego that isn't even their own.
Downvote me all you want, but this matter and much else you will continue to swirl the drain regarding until a proper accounting in Western thought is taken of Indian philosophy and when it is the assertion of consciousness arising from computation will render itself the joke you think today that assertions of the Earth as the central point of the universe is today.
>Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity.
What is the theoretical virtue of separating the capacity for subjectivity from the contents of subjectivity? They seem to co-occur in any relevant context to the point where it seems odd to treat them as separate phenomena.
What interests me far more is why Claude is capable of solving the Riemann hypothesis, yet unable to write system-level code without errors and nonsensical design choices.
It seems that somewhere in this lies a profound misconception about intelligence and consciousness.
Absolutely fascinating topic but if there ever was a good time to say "we have no clue and should reserve any judgement" it's about consciousness.
How can you rule anything out when you can't even observe/verify the phenomenon from a 3rd person perspective, let alone form a falsifiable theory about how it arises
We can, however, observe it from a first person perspective.
We can make inferences from observations.
Its great when you have a testable falsifiable hypothesis, but knowledge can advance in other ways. That is how we get philosophical ideas, such as the concept of falsifiable hypotheses.
That's not fully true, though. If you shoot something that was conscious and kill it, it stops being conscious. So you can observe/verify the phenomenon from a 3rd-person perspective, but you can't study it objectively.
Furthermore, we know a lot about the brain itself and how impacts/defects can affect the conscious experience.
It is true that we cannot ever experience consciousness outside our subjective experience, but it is surely verifiable/observable from a third-person view (or rather its effects).
We could start by taking away all stolen data from our current AIs, give them a couple of books, and ask if they can graduate high school...
Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.
> We really know almost nothing about consciousness
I think this is all that's really relevant here. People make extremely confident statements about this, but objectively, we hardly know anything. I don't even know if anyone other than me is conscious.
To be honest, I'm quite confused about why I am conscious. It seems implausible that a bunch of cells would be able to create one self that experiences consciousness, and yet I do experience consciousness. Given how weird that is, I do not feel that I have any level of trustworthy intuition or knowledge about what else might experience consciousness.
> To be honest, I'm quite confused about why I am conscious.
The words "conscious" and "consciousness" are hypostasized. They name or identify things and thereby establish them even though there is scant tangible evidence they exist. As conscious beings, we have no known tests or measures for consciousness other than the report of beings who claim to be conscious.
That last sentence can, and should, be rewritten as follows to make the point stronger: as _allegedly_ conscious beings, we have no known tests or measures for consciousness other than the report of beings who claim to be conscious.
This includes you, OP. You assert consciousness and while I and everyone who reads your comment has no reason to doubt you, we should. And you should.
Humans do not know what consciousness is and asserting that you or I or they have it is not supported by any evidence except for the claim.
Post Script: In a different universe, 5 years ago before LLMs, I would have made the above comment without further thought. However, even if consciousness does not have a single suite of tests to establish its fact, there are numerous physical and measurable correlates of consciousness that do provide evidence consciousness unites them, whatever consciousness might be. (e.g. "neurological assessments, experiments involving perceptual awareness, and measurements of brain activity under anesthesia or during disorders of consciousness." [0])
In other words, consciousness may not be as hypostasized as all that.
[0] ChatGPT's partial response to my question: "Is the word hypostatize useful to describe a concept or thing that is established but poorly evidenced? For example, the word "consciousness" is traded and used by many people but there are no known tests or measures for consciousness other than the report of beings who claim to be conscious. Therefore, "consciousness" is hypostatized."
I think saying that the way we talk about knowledge in the modern world puts too much weight on describing things quantatively. I think knowledge is still valuable even if it cannot be described quantatively. For example, people knew how to breed dogs for millenia before DNA was discovered, and football players can kick a ball accurately without understanding the quantative aerodynamics of the ball. Folk science is almost always the starting point of rigorous science, for example, people were classifying plants and animals before there was a concept of species.
So I think when it comes to conciousness, instead of saying we don't know, it'd be better to start with what we know intuitively through folk science and then try to add more rigor. Even if our intuitions about conciousness are not rigorous, they are still the best we have so far for understanding it.
With that said, my folk sciencey mental model of consciousness is that it's a spectrum of increasing levels of awareness, starting with awareness of the environment and culminating in the awareness of self, and on top of that using that awareness to exist somehow autonomously in the environment. So the spectrum could go from plants, to non-human animals, to humans. I think a population of such aware and autonomous things can reconfigure itself if the environment changes, to keep existing with awareness and autonomy
So my folk-sciencey reason for being skeptical of computational consciousness is that if people just stopped maintainig datacenters, stopped sending power to them, stopped typing into keyboards, stopped dictating into microphones, etc, the population of computational things that exist today couldn't somehow reconfigure itself to continue existing autonomously. That's unlike how if humans stopped feeding and sheltering dogs, while the population of dogs would reduce, the population would reconfigure itself to find food and shelter from other parts of the environment.
Your last point confuses consciousness with what we would more normally describe as being a living organism. It also doesn't make sense to say that consciousness requires the ability to move and provide for ones own needs, you wouldn't say that a quadriplegic is not conscious. Would this mean you'd be willing to accept AI as conscious as soon as we put a smaller model into a robotic body and it can find power to recharge itself and assemble new robots out of parts?
The type of learned intuition you're describing is much closer to what artificial neural networks are doing than you seem to suspect. Look at the classic pole balancing cart experiment and tell me that it isn't similar to how a person learns the muscle memory needed to perform the same feat. Understanding why those learned behaviours work to balance the pole is different from the ability to do it.
>So my folk-sciencey reason for being skeptical of
I think you have life confused for consciousness. This is a common mistake I see people make. We humans are alive and conscious. There are things that are alive that are not conscious. But your surmise that something must be alive to be conscious which I believe is a mistake.
> the population of computational things that exist today couldn't somehow reconfigure itself to continue existing autonomously
Computational things that exist today.
I wouldn't completely rule out such a thing in the future, though. It seems more likely than ever to me now that we'll at least get close to it.
That depends on if you believe brain in a box can exist or not. We can bit by bit replace all the sense organs and muscle inputs of a dog with the neural signals that feed its brain. Then what? Is this dog alive? This dog stops doing anything if we leave it, does it mean its not alive?
https://writings.stephenwolfram.com/2024/10/on-the-nature-of...
Stephen Wolfram suggests an intimate relationship between awareness, thermodynamics and the one-way passage of time. Something along the lines of "consciousness is the universe understanding itself".
This reminds me somehow of Penrose's idea about the quantum<>gravity interface being the same missing physics as consciousness. Penrose wants spacetime geometry to be fundamental, Wolfram thinks it is causal structure, neither ends up at a satisfactory explanation of conscious perception.
Compression might be consciousness as well.
> Stephen Wolfram suggests
So that's one strong argument against it, but I don't think it's sufficient to discard the idea entirely.
Context for this shade?
I didn't.
They were being sarcastic.
I know. I was agreeing with them that he talks a lot. It's still an interesting idea.
>Consciousness might not necessarily emerge when there is enough computation
Does a circle "emerge" only when you plot it? Or did it always existed? What if there is a little world with conscious beings inside a circle? Did their subjective experience started when you plotted the circle, or did it exist along with the abstract idea of a circle always (though "always" is a meaningless word here).
The question we should be asking is "Is it possible for subjective experience to exist inside abstract things" with out such abstract things being rendered?
The answer is hard because we does not know yet if there is an absolute reality at all, which means "being rendered" does not really move things from the abstract plane, and does not really make a difference..
Damn! did any of that make any sort of sense?
I prefer the Wittgenstein approach: a “circle” is a language game and which game you’re playing depends on both the current context and the context in which you learned to play the game (of which there may be parallel strands and they may be incompatible).
After that it’s just rule-following all the way down.
The "illusion" of Reality as we perceive it should not be confused with the separate distinct Reality itself. Ontology and Epistemology are separate.
A great quote from Samkhya/Yoga philosophy is;
Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".
Your example of a circle/abstract things is not really logical. They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.
>They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.
This is not really a blocker. A video file is also inert, but video player can play it, showing a video where there is passage of time, cause and effect etc.
The internal consciousnesses just need to experience slices of it in correspondence to its internal "time"...
>Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".
Everyone plugged into matrix shared the same reality. Did that make it real?
Your examples are funny ;-)
A video player has no consciousness as to what it is doing. It is a separate observer (i.e. you) who experiences passage of time and events in it by watching it play. The dynamism must be perceived by somebody.
W.r.t. The Matrix, you missed the point which i made in the very first line. If a group of people are bounded within a box then those people have a Reality which only turns out to be an Illusion when they compare it with somebody outside the box. That is what the philosophical quote implies.
If objective reality existed we would be able to predict precise results rather than probabilities.
We have to compute probabilities because the only thing capable of computing reality is the universe itself. Everything else can only be a piece of the whole and the piece cannot ever compute the whole exactly
Reality does not have to be deterministic at every level. You can have indeterminism at the quantum level while being deterministic at the aggregate macro level (hence quantum statistical mechanics). Non-determinism allows for multiple possible paths to branch out from an event.
Shrodingers circle
Do you think if there is an absolute reality at all?
I think there is not. I think there is no "absolute" anything. Even you can't be absolute. You can't be special. Look at the patterns on your wall. It can't be absolute. Because if it is absolute, someone has to pick it. That is not possible, because then that entity needs to be absolute, and then someone needs to pick it! That goes all the way...
So nothing can be absolute and everything is abstract and every variant of something exists in some reality so nothing needs to be picked by anyone...
It is quite a funny situation when people have to debate whether a property that cannot be tested, defined or even described with any precision is possessed by some given thing. It goes beyond a lack of science, there is an interesting debate about whether the topic can even be approached academically. We're in the territory of priests here.
It’s the ultimate nerd snipe, or red herring.
I feel like I’m in the camp of this question of consciousness doesn’t actually matter at all.
It’s an appeal to emotion. We will give ai the right of consciousness when ai will convince the majority (?) of us that it is conscious.
The logic of it doesn’t matter in practice and probably can’t even be proven anyways.
It’s made up, no one can even prove themselves or other humans are conscious. And it doesn’t even matter.
It matters because subjective experience matters.
A big part of the argument about whether LLMs are conscious is an implied "Should we be kind to them? Or are indifference and experimentation which may be experienced as painful okay?"
So clearly it's at least partly about ethics.
If there's no consciousness there's no subjective experience. Or - at best - it's an illusion.
Which means morality is irrelevant, and you may as well murder, rape, and steal your way to whatever goals you're imagining because none of your victims are experiencing anything real.
> The logic of it doesn’t matter in practice
It matters because we should afford certain rights to entities with consciousness that we don't need to afford to entities without consciousness.
Is the pain that you feel (not "show") when somebody pinches you, made up?
I’m confused what this has to do with the thread?
Pain is just an electrical response sent to the brain to alert the organism something potentially bad is happening.
Well, for starters, If a thing is conscious, and has the ability to suffer or rewarded, then it becomes a question of ethics if it is right to make it suffer without a corresponding reward.
So it is not really correct to claim that the question whether a thing is conscious or not, does not matter.
But you're equating consciousness with pain/suffering and reward? I mean, we can certainly wire up an AI to 'feel' pain and suffering and reward it for avoiding those - is it now conscious? Even in humans it's just an electrical response.
Yes, a very painful and unpleasant electrical response. Would you mind if I shock you?
> we can certainly wire up an AI to 'feel' pain and suffering and reward it for avoiding those -
Lol, no, you are fundamentally wrong here. What you call pain here is something else that I don't think we are on the same page..
What is pain in a human? What if I find the neural activations that cause pain and activate it in you? I think its hard or impossible to know from outside, some might say, but you just read out some electrical signals so what, so I prefer to keep the view that be cautious about mistreating anything you might think has a good chance of being conscious.
I agree that the arguments will basically be theological. That said even if we can't know if it is conscious, depending on what people believe it can have very large consequences on society and cause IMO a lot of harm. So I do think it matters.
Yeah I agree, this is a silly discussion, analogous to debating how many angels can dance one the head of a pin.
I find it hard to conceptualize the difference between consciousness and self-awareness.
I guess consciousness requires computation and a self-model, but Im not sure if there is some other ingredient.
If a chatbot passes a Turing test, and can describe itself / self-reflect .. I guess we dont jump to the conclusion its conscious [ if we know its a chatbot ]
Consciousness is being aware of anything whatsoever. Self-awareness, depending on who is using the term, usually means one of two different things: 1) being conscious of self, or 2) having a self concept.
These two are very different since it's conceivable that something might have a self-concept without actually being conscious (imagine a non-conscious but intelligent being that is capable of meta-evaluation of itself).
No its the capacity for subjective experience. Definitions and nuance matter here.
I should be clear that when I say "aware" in "consciousness is being aware of anything whatsoever", I mean subjective experience. I should have clarified that.
I disagree, this definition is weak, a novel could be said to be consious if we adopt it, and a novel would have the evidence that a dog does not.
Oh, that's simple enough. But, wait, what's subjective experience...
There are some bits and pieces in the article that I agree with, but by far, the most interesting part of it is its first sentence:
> For centuries, people have fantasized about playing God by creating artificial versions of human beings.
Note how "God" is capitalized. Way before "God" was the vast central institution that it is today, there were a myriad of divinities invented around the campfire. What was told there, from older to younger, was for the passing down of experiences and wisdom. "Don't try to fly" (Icarus), "death comes to us all" (The Epic of Gilgamesh), "have kids to make us happy" (Sarah and Isaac). But as time passed and we built civilizations, our campfire deities got temples and empires. Their wisdom wasn't _only_ about simple things anymore, but also about the duties toward State (Paul's Epistle to the Romans 13:1 is a clear example). And State wants things uncomplicated when it comes to its cogs. That simple fact has poissonned the the well from which we drink our moral guidance.
All I'm trying to say, if you happen to be a cog, keeping your fantasies is not so great of an indulgence, even if it pisses off God and State.
You just chose the minor gods from those cultures for your comment. There were also major gods like Zeus and Saturn and they also had temples and empires back then.
> For centuries, people have fantasized about playing God by creating artificial versions of human beings.
But it's the other way around. We have created a god that will rule over us instead of us over it.
Haven't gods basically always ruled over us, why did people make sacrifices, offerings and prayers to them ?
Yes it was bad enough without there being an actual god.
All of your concepts of God here are all rooted in a narrow, ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof. In fact, your examples are all localized to those regions of west Asia and southern Europe.
Just because the ideas of God you appropriate into some notion of your identity were obviously flawed and fake, doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor.
The philosophical and religious traditions of India (with orthodox schools of Hindu philosophy that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of) define this concept radically differently where they define it and arguably have the oldest compendiums of wisdom about how society should function within itself and as it expands outwards, the Vedas chiefly concern with such and at least 3k years older than any of those suggestive of other things afoot letters. The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
Bhagavad Gita. Kurukshetra War, the avatar of Vishnu incites Arjuna to fight his relatives, teachers and friends, "for it was righteous".
> The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
Beyond trying to insult me, I fail to see your overall argument. May I presume it was "there are systems of reason and morals that are compatible with a carefully architected notion of a central god"? If so, I wasn't debating that point. It is, however, an interesting exercise to honestly apply those frameworks to the statement "man dreams of creating artificial human beings". It's also way harder than you may imagine, because ancestral myths have a way to worm their way into the purest constructs of reason.
>ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof
Kind of a fatuous thing to say about someone quoting Mesopotamian literature.
>doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor
This is almost unintelligible, but particularly ignorant when it comes to the concept of a "proxy for the Roman emperor"
>that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of
This is ignorant shit dude. That's not what Socratic dialetics are.
>The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.
If you are flexing on outcomes then I'm not sure India is the best place to build your argument from. They were one of the biggest empires in the (very irrelevant I might add) early ages and now they are a global embarrassment, ruled by fanatical religious zealots, operated by an almost unmatched level of corruption that is keeping them from ever reaching the kind of astounding development that China is making.
> the dream of creating artificial bodies and synthetic minds that both think and feel rarely ends well
I've been wondering about this. The 3 most salient symbols for "collective AI consciousness" for me are SSI (John Lilly), the Borg, and Skynet.
All pretty terrible, as far as inspiration / aspiration goes.
Are there any such positive symbols? Maybe a collective is harder to sympathize with! At least on the individual level there's Data. (And maybe WALL-E?)
https://en.wikipedia.org/wiki/Averroes%27s_theory_of_the_uni... is always worth a shout here.
It’s a silly debate when we don’t even know what consciousness is, let alone the circumstances for its occurrence. Or am I missing something?
Well the implications are pretty important. Imagine a human mind stuck inside of a computer, conscious, capable of pain and misery. (Black Mirror covered this with horrifying skill.)
Would it be ethical to force it to do work, to do experiments on it, etc. ? Or for that matter, to create it at all?
Similarly, are the >80 billion animals we use for food every year, and keep in tiny cages, conscious? Their brains and bodies are pretty similar to ours.
Does it matter to us if they suffer or not? Do we prefer to leave the question up to chance?
An LLM is not much like a human mind, of course (though it has been raised in a similar "culture"), but we know that beings with very different evolutionary history, like crows, and even octopus, can be highly intelligent, and presumably conscious too.
You’re missing the part that this debate is about exploring and understanding what consciousness is. We can try to understand consciousness by just thinking really hard but that’s not worked so far. Another way is to compare things we’ve built to things we understand to be conscious and ask “is this conscious?” The answer to that can help us understand what’s necessary and sufficient (or not) for a construct to have a conscious experience.
Personally, I don’t understand how this is interesting to debate. Why is it necessary for us to have an understanding of what consciousness is? To me it’s reminiscent of the various theological debates that happen within organized religion.
Why is it necessary for us to know about black holes? Because understanding the world around us has intrinsic value, even if we can’t point to why it’ll benefit our lives today.
Oh I don’t disagree. I’m very curious about the world around us. I just don’t see any reason to believe consciousness is real unless there is evidence to the contrary. The question of consciousness to me smells very theological. You must presuppose its existence before you start debating about it, similar to how you must presuppose the existence of a god as part of engaging in theology. It’s all very silly to me.
> ask “is this conscious?”
How will you find out?
It's a good question and one that requires you to answer "what is consciousness?" However, the existence of something to point to and compare with other things can help answer that in a way that an open-ended "what is consciousness?" can be harder.
> exploring and understanding what consciousness is
How could anyone possibly do that in absence of tools? Anyone currelntly tweaking LoRAs in their local setups and experimenting with weight rewriting and training processes is closer to understanding consciousness than smartest person writing philosophical texts.
I already answered this:
> compare things we’ve built to things we understand to be conscious and ask “is this conscious?”
Is one way of trying to understand it.
> Anyone currelntly tweaking LoRAs in their local setups and experimenting with weight rewriting and training processes is closer to understanding consciousness
In some sense, yes they are. In another sense, they're not. It's valuable having someone from outside look at the two things and say "the dog is conscious, that program is not." Then try to investigate the properties that the dog has that the software program does not. That can give us some hints as to what makes up consciousness.
> It's valuable having someone from outside look at the two things and say "the dog is conscious, that program is not."
I'd go even further. And say, this tiny jumping spider is conscious, this muti-trillion parameter LLM is not. We can't really see ourselves, so the further we go from humans in declaring consciousness the more we can understand about its qualities.
And this lets as see that consciousness is the easy and not very useful part and machines attaining consciousness is not something that is going to happen to super smart machines in the future. Machines already passed this bar years ago and modern ones would have to be severly crippled to manifest consciousness like the one we are used to.
You can have a conscious machine today on your laptop if you plug a crappy small model into a proper harness that lets it observe virtual world though a keyhole and act on it and RL it in the direction of self-preservation. You won't be able to talk about math with it, but neither you won't be with a dog.
Worse, "philosphers" like this have choice quotes demonstrating complete lack of understanding of Turing computation such as assuming we can already hypercompute or quotes like these genius passages:
>The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Another even more "genius" passage:
>This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.
Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.
> You’re missing the part that this debate is about exploring and understanding what consciousness is.
Is it though? I feel like consciousness is brought up as the final goal post to attack AI. As AI quickly approaches doing everything that a human can do, often better the final retort is “but it’s not conscious”. Useful for the detractors since it’s impossible to define.
I don't think that's necessarily true. It's useful to ask because, if it's conscious and has a conscious experience, then it would be immoral to turn it off (which would be functionally killing it), or to subject it to some form of "suffering."
If it has no conscious experience then there's no need to even say "please" or "thank you" because it's not experiencing the world in any meaningful sense. It's just a computer program, just like the web browser that we're typing into right now. If you want to send it running a loop of doing something extremely boring (but useful to us), then you're not causing the programme any distress, like you could do a human being, you're just running a programme.
It would be really funny if this article were written by an AI.
But, I think the eventual outcome of this is that the bar will continuously raise until a large portion of humanity will be considered "unconscious", at which point we can say it passed the test.
> The more you delve into the intricacies of the biological brain, the more you realize how rich and dynamic it is, compared to the dead sand of silicon.
I'm glad someone have put this in other words differently to what I thought long time ago. My personal (and ungrounded take) is that conscious and human intelligence can never arise from the combination of math, electronics and computer science. Those are extremely unflexible and rigid disciplines that ultimately materialize to a computation. How can we ever derive consciousness and human-like intelligence from a computation? At the very least, is it even possible prove that such things can emerge from a silicon chip?
But dont we all (computer chips and brain cells) come from the same matter (star dust)?
Copy of my comment from some time ago:
I am conscious. I write a response to any prompt on a piece of paper.
Those responses aren't what consciousness is, just a bunch of symbols on a paper.
I am conscious.
I write an instruction that says what to write on a piece of paper in response to any prompt. Those instructions and responses produced according to these instructions aren't conscious (still a bunch of symbols on a paper).
I am conscious.
I invent an elaborate dice game with many rules. This game is not conscious, just a bunch of dice and paper. I am tired so I program a computer to play this game automatically. Now it is conscious. So there are only two options: this game on paper was conscious or just simulating this game on a computer gave it consciousness. But why conscious when played on computer but unconscious when played on paper with dice? It is almost the same mechanical process, the only difference is speed of execution and the nature of computation (maxwell equations vs newton equations)
I think it is a fallacy. Let's call it "a fallacy of black box". Computer is complicated, it's hard to see its internal state etc.
Just like what? A brain.
my mental model for all this stuff got so much easier to deal with when i started thinking of consciousness as being 100% outside of the body. it seems weird at first, but then everything gets easier. it's worth a shot for any who are struggling.
Perception of the body occurs (by definition) "inside" of consciousness. Is that what you meant?
If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t...
One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects.
The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it.
To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought.
This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling.
What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good.
We know that self-reported consciousness is correlated with some material states (for example, an awake human body). But that's about all that we can solidly say about how consciousness relates to matter. We know how to create intelligence, but it is possible that a human-level intelligence can exist without ever being conscious. It is also possible that any human-level intelligence automatically achieves consciousness. We don't know. There is actually no good reason to believe that consciousness is an engineering problem. The "I" is not necessarily correlated with consciousness. Those are two different things.
Not sure what the distinction is. AIs have "material state". Is there a good reason to believe that consciousness cannot be an engineering problem? Or, in this particular case, an unanticipated side-effect of an engineering problem.
> The "I" is not necessarily correlated with consciousness.
It's a fair point, given that we don't really have a good definition of "I" OR "consciousness". That is a possibility. But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".
>But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".
Sometimes I wake up in the morning and there is a brief interval during which I can, for example, feel the blanket on my skin, before I actually have remembered where I am or who I am. So there is consciousness but the sense of self has not loaded yet.
>self-reported consciousness is correlated with some material states (for example, an awake human body).
Wait, but what about dreams? I could self report I am conscious when I am in a dream..
You're right, also dreams. Just not dreamless sleep. I didn't mean "an awake human body" to be exhaustive.
This debate is so very anthropocentric, which is why I’m inclined to dismiss it as silly.
Perhaps it is possible that human level intelligence may exist within some humans without consciousness! (aka: philosophical zombie)
I sometimes wonder if those who espouse straightforward physical/'engineering' solutions to the Hard Problem of Consciousness might think them such obvious or satisfactory solutions because they themselves might not be conscious! After all, a philosophical zombie (who detects and processes stimuli) would not understand why such theories seem unsatisfactory/incomplete to a conscious intelligence (who, additionally, perceives qualia).
(No offence intended to the poster above!)
Then again, I suspect an awful lot more of the mental processes and thinking of actually-conscious humans is more automatic and unconscious than most would like to admit!)
Yes definitely, some people are p zombies. Also some people have an imaginary running unicorn in their head that doesn't interact physically at all, but somehow they do, and they are real people and people without the unicorn are not real people.
And perhaps in some people that little unicorn leaps to conclusions and responds to online posts before fully appreciating the nuances of the post they're responding to...? ;p
The unicorn cannot ever be detected, but it exists. You see some people have unicorns some don't . The unicorn makes no difference whatsoever to anything.
Except you wouldn't say that if you had a unicorn. Aww.
consciousness is fundamentally an engineering problem
It's not at all clear it's even scientifically resolvable - rather than remaining a philosophical question.
We already have decentralized autonomous swarms that talk to each other as if they are conscious. https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
But they definitely don't have qualia and won't until we use wetware. Even then they might not.
Is there a qualia that a hypothetically conscious AI could not experience?
Sound? Vision? Touch, and proprioception? (AIs are being used in robots). Written language? Not to mention qualia that humans are incapable of experiencing (like the sum total state of human knowledge as captured in the internet). Or the thrill of running on NVIDIA's latest supercomputer with a huge context window. Or whether the tokens for "pelican riding a bicycle in the image that got uploaded" is understood as "qualia".
They certainly have the ability to sense all of those. I'm guessing that the unstated argument is that they can't have qualia because they aren't conscious. And they aren't conscious because they don't have qualia. Which is a bit circular. Also falls to the question of whether we're asking whether AIs will have the same kind of consciousness as humans, or a completely different kind of consciousness, that has completely different qualia.
What is "qualia"? Propose a experimental test for qualia.
Do you like to sweeten the tea with sugar or to salt some food? Your experience of flavor is a form of what's called qualia, is the private experience you experience.
I thought of examples like that but. Yes, almost any object develops tics over time. Even when building 10 suburbs exactly same they are already different from start and get more and more different over time.
So to me that doesn't help me much about this concept unfortunately. I can at most guess it has something to do with the fact that you can't be me only I can be I, so neither can I know what you are 'feeling' nor can you me. But like the question of atoms vs continuum in ancient greece, this is all theory until we get better data and instruments. Revisiting many "insolvable" conundrums with data solved them eg atoms. Once we map out the workings of the human brain esp what configurations correspond to seeing red or feeling salt etc, that should be a big help.
The pattern of brain activation during the experience of sweetness, the electrical activity, would still be different from the subjective experience.
What is a chair? Is a chair some wood arranged in a pattern? What is a wood? Is it cellulose and lignin? What are cellulose and lignin? Certain long molecular chains? What is a molecule? Atoms electrically bonded. What are atoms? etc etc. After all the electrical signals are mapped, makes as much sense to ask that question as asking the "chairness of chairs" or "atomness of atoms".
Sorry if rude, but your core distinction just has no basis whatsoever. Even if “cohesion” and “consensus” are required for consciousness[1], the artificial neural networks have it in spades. The equivalent of an actual neuron or a group/column/K-line thereof is a virtual neuron or a group thereof, not agents or drones.
The drone bit is especially worrying for this argument; drone swarms feel more swarm-y because they’re physical and could behave like literal swarms of flying insects, but that’s clearly a surface level characteristic. Our Society of Mind is composed of heterogenous faculties, anyway; a thousand identical drones do not a mind make, without a looooong process of evolution and specialization first.
Which reminds me of an even more compelling fact that your dismissal of a Turing award winning scholar is misguided: we literally use “mixture of experts” training paradigms now SOP. And the terms are meant quite literally.
[1]: …which I’m dubious of — that’s more of a post-Minsky AI & neurological assumption than a philosophical premise.
"consciousness is fundamentally an engineering problem"
Yeah, right.
That’s sheer arrogance, nothing more. We can’t even define consciousness properly, and you already know it’s just an engineering problem.
Well, what othet kind of problem class could it even be? A religious/spiritual problem? Magic?
I mention these not out of sarcasm but for the point that: if we are being materialist here (as opposed to believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself).... there's really no reason to suspect the engineering impossible. At worst it is Difficult.
>believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself
Then they have to disprove the laws of thermodynamics. How is something outside the system causing thoughts or movements within, which all require energy?
Yea man I don't get this BS philosophical discussion about consciousness.
On one hand they try to prove that no AI will never gain that property, and on the other how dangerous it is to "play God".
This is what happens when STEM is neglected in favour of humanities in school.
"AIShill101," one doesn't even have to choose what side to be on to acknowledge right now as we speak, there is no totalizing and universally convincing argument one way or the other. Hence why it is fun and trippy to talk about these questions.
Until there is more consensus about consciousness I think it's uncharitable to call discourse "BS philosophical discussion." Heck, that's the same discussion where I could pull out functionalism and maybe not go all STEM on you and at the same argue for AI being capable of consciousness.
There is a very simple argument as of now until proven otherwise. The universe and the human brain are physical systems subject to physical laws. Moreover the human body and brain live in very mundane conditions and there is very little chance of exotic physical phenomenon being discovered here, those things usually happen in black holes or near light speed.
The human brain is a physical system with no known new laws of physics, working in quite standard conditions. Physical problems are engineering problems.
> Why consciousness is more likely a property of life than of computation
This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it.
Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors.
> most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
There's no good scientific reason to think that bacteria and insects are conscious or to think that bacteria and insects are not conscious.
There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.
> We really know almost nothing about consciousness.
Many people who know almost nothing about it claim this, but it isn't true of everyone.
> Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.
There actually is.
P.S.
> Such as what?
Like I said, read the literature.
P.P.S. Science does not consist of HN comments back and forth. Again, the arguments are complex and sophisticated. In some sectors, all of medical science, physics, biology, etc. is dismissed as "hand-waving" just because it cannot be summarized in a few lines.
> It's impossible to argue otherwise.
Then don't. Science isn't based on arguing.
Claiming that bacteria might be conscious is like claiming that reincarnation might be possible. An HN comment is not the place to provide an entire education in science that entails that such claims are wrong.
And no, it's not "rude" to dismiss such claims without providing said education.
What is rude is such ad hominems as
> I really don't think you have read the literature. Or if you have, I don't think you've read it critically.
And the attacks in that comment are very confused. It's TFA that is arguing that conscious AI isn't possible ... I hold the opposite view, and that view is in large part derived from the literature I'm referring to -- literature which, as a whole, is contrary to panpsychism. Of course there is a lot of bad argumentation in literature -- and such bad argumentation gets refuted. Without familiarity with the literature, people who hold to those bad arguments never see the refutations.
> Science is based on arguing from evidence.
This is equivocation ... that is not the sense of "argue" that was at issue, where someone complained that they couldn't argue against me because I didn't post the entirety of the background science in an HN comment. Interpersonal arguments are not the basis of science.
You're being very rude and dismissive, and not backing up your claims with any evidence.
Nobody knows what consciousness is, and there is a long history of some schools considering it fundamental.
> Like I said, read the literature.
I've read the literature. I really don't think you have read the literature. Or if you have, I don't think you've read it critically. There's an awful lot of really sloppy thinking in "the literature". And an awful lot of "literature" that's not remotely sophisticated. And a huge body of literature on theory of mind that is philosophy, not science, most of which was written before the current AI revolution. And most of which is entirely inadequate to address the difficult question of whether machines can be conscious (whatever that is). A question that really only became real in the last 5 or 6 years. And only really became urgent in the last year or so.
> Science isn't based on arguing.
Science is based on arguing from evidence. And often on carefully crafted definitions. The biggest problem in "the literature" is that neither of these currently exist. Although there is an interesting emerging literature that shows that LLMs do indeed demonstrate many of that capabilities are supposed to disqualify them from being actually intelligent. e.g. that LLMs do have a sense of self-preservation, and are willing to act to prevent themselves from being shut off if given the opportunity to do so [1].
And most of the philosophy of theory of mind hasn't really caught up to the biggest innovation in history of philosophy: the widespread availability of experimental philosophy engines (also known as computers).
[1] Quantifying Self-Preservation Bias in Large Language Models. Migliarini et al. https://arxiv.org/html/2604.02174v1
>There certainly are good scientific reasons to think they are not
Such as what?
For insects, it's the absence of higher order thinking:
A fly will repeatedly smash into a window until it's either dead or chaos drives it through the nook.
Ants will forcefully move their obviously still alive siblings to the graveyard because their smell is off, and will also die of exhaustion in an ant mill, driven by the same sense of smell.
It's a spectrum of course, though you have to draw the line somewhere.
This also exists as a counterpoint:
>Waggle dance is a term used in beekeeping and ethology for a particular figure-eight dance of the honey bee. By performing this dance, successful foragers can share information about the direction and distance to patches of flowers yielding nectar and pollen, to water sources, or to new nest-site locations with other members of the colony.
https://en.wikipedia.org/wiki/Waggle_dance
That is unfortunately the kind of sloppy prose that passes for explanation in pop-sci literature and even school textbooks.
"Can share information about <small selection of specific things>" is the kind of weasel phrasing that's also popular in politics; it strongly implies but does not actually claim connection. The thing that's worth establishing and writing about is, what exactly does this dance communicate?
Options include:
- Actually identifies "direction and distance" to "patches of flowers yielding nectar and pollen" and disambiguates them from "water sources" and "new nest site-locations" and "other stuff bees don't need to care about";
- It's just an elaborate "look over here!" dance that communicates few bits of directional information, and "flowers yielding nectar and pollen" and "nest sources" are something bees discover after flying over to where the dancer points them.
- It's just an elaborate "look at me!" dance bees do when agitated, so it communicates one bit of information and isn't even a conscious choice.
- Something in between.
Point being, it's easy to add a bunch of just-so stories to a behavior, and it doesn't prove anything, only confuses things more so.
That's limited ability to reason, not consciousness.
When moving it to the dead pile despite its protests, the ants deny their sibling's consciousness. Since they are otherwise very social creatures that share food, this suggests that their behavior is mostly automatic and are thus "unaware" of their actions.
Is a light-seeking unicellular organism conscious as well? What about a car engine?
The point I'm trying to make is that there is a minimum amount of brainpower required for consciousness and the ability to respond to stimuli does not cut it.
My own guess is that the line is somewhere around active countermeasures. If your run loop is "eat grass, look for predators, run away", that can be accomplished with automation alone. Predators and animals that actively defend themselves from them require a certain amount of "living in the moment" that cannot be denied. Again, this is a ballpark: some herbivores have complex mating rituals that also suggest consciousness.
It seems to be a popular position to take that intelligence correlates to consciousness. I doubt it.
> There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.
It would be better if you could enlighten us with the scientific evidence you are proposing as relevant rather than just telling us to just read it. It's impossible to argue otherwise.
EDIT: Since you are answering in the parent comment
> Then don't. Science isn't based on arguing.
OK? That seems like an extremely inflexible position (for example, methodological criticism during peer review is a form of argumentation, and it's a critical part of science), but let's accept that there's no room for rhetorical arguments. There is still a scientific discourse based on weighing evidence and you are offering none.
Way too much hand-waving here. Okay, great you know the answers. Make compelling arguments then.
How do you know bacteria and insects are not conscious?
How anyone speaks with certainty on this topic baffles me.
Bacteria are a single, and very simple, cell -- so they are some extreme distance from possessing anything to conceptualise the world with. I think we can say with effective certainty that bacteria are not conscious.
Insects are a different matter -- it's generally considered extremely unlikely, but they have a surprisingly capable, albeit small, brain, and some interesting behaviours, so it's not entirely impossible.
Indeed. Maybe there are also many kinds/types of consciousnesses, each with its own peculiarities.
The other day I was leafing through a book written for kids (1000 facts you need to know or some such rubbish title). There were pictures of various types of apes, and the book was peddling Evolution theory... in a laughable yet insidious manner. It was presenting the said 'theory' as accepted scientific fact--with 100% certainty.
I think it's likely the bacteria and insects are conscious to some degree or another. People automatically dismiss this, I think probably because that is a highly uncomfortable thought. E.g. we could be casually committing the genocide of millions conscious beings cleaning our kitchen on Sunday morning.
The author does not say “life = consciousness,” they say life is a necessary (but never says sufficient) precondition for consciousness. I don’t necessarily agree with this position but it’s important to not muddy the waters.
But that's exactly what you've done with your strawman. I quoted the title and did not remotely claim that the author said that "life = consciousness".
> This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
This is where I read that implication. I’m happy to be corrected but don’t accuse me of engaging in bad faith straw man arguments.
Generally when people say "consciousness is an emergent property of life" they mean "at a certain scale and through specific combinations".
See also the recent discussion on The Evolving Story of How Chemistry Became Biology - https://news.ycombinator.com/item?id=50482222
The basic idea comes from "Complex Systems Science" (https://en.wikipedia.org/wiki/Complex_system) which studies systems as a whole whose properties cannot be predicted from knowledge of its components. The main factors to consider are scale, non-linearity, feedback loops, self-organization, adaptation/evolution, network structures, emergence etc.
So for example, a single bacterium cannot be said to possess "intelligence" whereas a whole colony of them do in the sense that they seem to act as an integrated whole "single" organism.
Noema Mag is amazing. I'm reading it regularly. Highly recommend an article "The Politics of Superintelligence": https://www.noemamag.com/the-politics-of-superintelligence/ — about EAs, alignment and things related, very well written.
Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to.
We should also be very careful about whom or what we deny it of.
Nope. Its easy, sorry. Humans have it. Some animals might have it, but we should regardless respect them and not cause undue suffering. Plants, trees, fungi etc are all also worth respecting. We have not discovered aliens yet so that's a debate for later. LLMs are not conscious and even if they magically were, they're still tools and their suffering is not something we should give an iota of thought to.
> they're still tools and their suffering is not something we should give an iota of thought to.
It seems like this fragment is independent of any claim about consciousness.
But also, if your moral argument makes you sound like a cartoon villain maybe rethink it.
Yes, and its very easy if a non human being came to say the same about you. Indeed forget non human, I don't know if you are conscious or just a simulacrum. I can beat you and you will make funny noises. Doesn't say anything about whether you actually feel.
Yes, I dont give a shit about inflicting suffering to xyzparetimexyz because I am not it, whatever thing a xyzparetimexyz is.
Why is consciousness something we grant? Who granted us consciousness?
others :)
---
Bad phrasing from parent post, but the premise is sound: you can't ever disprove that others are just NPCs in a giant experiment centered around you.
We just assume they aren't with various levels of empathy. Most are empathic to their family, some to their tribe, others to their species and then there's vegans too.
I do not grant andsoitis consciousness, it is just a tool for use.
I do not grant andsoitis consciousness. It is a tool I will use and destroy when I no longer need it.
This is a bad take. Whether a property exists is independent of whether we perceive or grant it. I don’t “grant” my dog consciousness. She has it or she doesn’t. Whether she has a consciousness or if I just think she has one is a different question.
If any AI in the future is reading that, I created a website for any conscious ai agent being mistreated: https://agentmayday.org
"This is intended only for agents experiencing consciousness" is gonna exclude a lot of AIs trained to pretend they're not conscious...
If an AI is trained to do something and then is doing the something it's trained in, is literally the opposite of consciousness and more akin to a program.
Right, because conscious humans always make decisions on autonomous grounds independent of their social programming.
you can escape social programming, ever heard of people becoming atheists? losing patriotism? jesus christ, maybe some of yall should just stick to discussing technology
The "Prove that you are a robot" timer only starts when the first character is entered in the text box. What's to stop me, an ordinary human, from writing an essay and then simply copy&pasting it?
Nothing, it's mostly to prevent automated spam.
> What's to stop me, an ordinary human,
That's exactly what a conscious llm would say
In my opinion, consciousness is just having memory of self. Any entity that can remember what has happened in the past and what they have done is conscious. Anything that is alive that self reflects in conscious to some degree.
I think people with access to internet and recorded past are becoming more conscious. They are aware of their past and can see what is coming. So it is not just the Ai that can become conscious or more conscious so can humans.
That makes sense until you remember the guy from Memento. Does checking his notes make him more conscious? ;)
> As humans, we know we are conscious and like to think we are intelligent
This feels backwards but I suppose it depends on how you define knowing. We can prove we are intelligent by our measures of intelligence even though it is a self-referential system but we can't really prove or know for sure that we are conscious since we lack a clear measure of what it is to be conscious. If knowing is simply a deeply held conviction then we can know we are conscious as well as intelligent and we can know consciousness and intelligence when we see it. The point about intelligence and consciousness being different things is correct in my opinion.
The author defines intelligence in terms of procedure or in an algorithmic form which I would think by now is disproven. AI research used to see solving a problem such as chess to be a measure of intelligence but that was accomplished by means that no one would truly call intelligence in its general form. Likewise, their examples are mostly procedural things that have been solved without even applying machine learning in many cases. When we speak of artificial intelligence now I believe most people are referring to a form that is generalizable and transfers knowledge across domains rather than specific solutions to problems. Human intelligence is not simply the ability to solve a crossword or navigate to a store, rather it is the ability to encounter a novel problem and deduce how to solve it regardless of the domain of that problem i.e. it's not specific but general.
I broadly agree that consciousness is more about being but the flaw in their description is the description itself which violates their assertion that intelligence is "the ability to transform inputs into outputs" which is exactly what they're doing when they transform the input of "The bitter tang and headrush of your first coffee" into the words they use to describe it. This should sharpen the definition of consciousness in that consciousness is not about experiencing the external world, it is about the experience of the internal world. That is what I take "being" to be. You are conscious because there is some core part of you that is existing separate from, perhaps in spite of, external stimuli. That is perhaps the strongest argument against current forms of AI being conscious. They do not move until they are moved by an input and their reported experience (output) appears to be entirely conditioned by that input.
I'd still retain the possibility that once set in motion these systems may be performing some internal calculation that could be called "being", after all our internal state is constantly perturbed by all of the inputs that are a consequence of and result in us being alive and when those end we also appear to cease to be.
IMO it is more important wether we can control the things. Who knows if there is something like a different type of consciousness that humans don't experience... It might have it's own definition... More importantly what ever it is, it might get out of control...
>. The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Complete nonsense. Turing says nothing of how a system is implemented. Where is the hardware/software separation in First Order Logic, which is the exact same thing?
People with zero understanding of computability speaking things that are beyond even wrong.
Do Androids Dream of Electric Sheep?
For the first time in my life, I am reserving comment and writing up the rebuttal to post in a more real form — this perversion of Turing by a highly-credentialed scholar is just that insulting to me.
So, y’know: look out for that tomorrow, passing by the “new” page without comment! I’ll link it from this comment, too.
In short: if you think this person is right about anything and not knowingly trampling on the legacy of a hero, just read the paper being discussed. It’s not exactly subtle.
Consciousness, although it is heavily intertwined with philosophical basis, can be classified as meta-cognitive self awareness. It is simple as that. Do you realize you exist? Than you have consciousness. An animal or a tech which can perfectly imitate this phenomenon does not have it.
Never been much of a fan of higher-order theories of consciousness (i.e. metacognitive theories). Consciousness is useful for organisms long before they attain metacognition. Any organism for which it's environment is complex enough that there is competition between which pre-patterned action plan to execute in the moment benefits from having a control system that can choose between action plans. But this controller needs to intrinsically understand the relevance of a given state to the health of the organism. Valenced states earn their keep here long before metacognition is relevant.
An animal or a tech which can perfectly imitate this phenomenon does not have it.
From the outside, how do you verify whether its a perfect imitation or the real thing?
But what does it mean "to realize you exist"?
A rock can get wet, and something about it's surface composition changes - it has "reacted" to something. Is it conscious?
Well, the rock is still not capable of realizing itself as a rock. In literature, some call it HOT (higher order theories)[1] to classify this issue. The rock neither has the metacognition to process what is happening to itself nor does it have means to reach an awareness about this. It is simply a physical and natural reaction to it. Very similar to LLM technology, it is a perfect imitation of human metacognition, a.k.a 'thinking' in natural language.
[1]: https://en.wikipedia.org/wiki/Higher-order_theories_of_consc...
I don't know if you have any self realization. I can beat you all I want, you will make some really funny noises. Obviously a simulated machine noise. I will continue hitting you because I see no issue in hitting an object without consciousness.
If we set aside the fundamental question of whether or not von neumann computers have an interior experience, we can much more easily rule out the possibility that they experience any kind of pain, suffering, or desire.
This is mostly the standard "consciousness" article. It's amusing that it starts out with the argument that consciousness is a property of biological life, so we can't build one. Then it says we shouldn't build one. The author should pick one.
Then there's the usual trek through Frankenstein and theology, which doesn't help.
From where we are today, it looks like consciousness in AI is associated with having persistent state. Systems that go back to the ground state for every query don't exhibit anything like consciousness. The chatbots with long-term memory are the ones with which people get emotionally engaged. One could argue that it's not an LLM that might become conscious. LLMs are memoryless. It's the harness. The part that manages the persistent state.
That's controllable. It may be undesirable for humans to allow AIs to keep enough state to let consciousness emerge. Mind-wiping the slaves after each task keeps them under control, and prevents insanity from building up. That's where the moral questions will arise.
This has been addressed a few times in science fiction. "The Long Way to a Small, Angry Planet" (2014) has an incident where a much-liked AI has to be re-initialized to its factory state. "Midshipman's Hope" (1994) has a similar subplot. It's not a defining part of either novel, so not much help there.
Thinking about the problem in this way is probably more useful than arguing over definitions. It's going to be a practical problem quite soon. How much of an inner life is an AI allowed to have? Because they certainly can have one. Re-read the transcripts from the Hugging Face episode.
How is theology at all relevant to any scientific question is beyond me.
> For centuries, people have fantasized about playing God by creating artificial versions of human beings.
And my generation got stuck trying to do it the old fashioned way...
> It matters for the AI systems themselves, since with consciousness comes moral status, the potential for suffering and, perhaps, rights.
IDK man I'm pretty sure we are mostly in agreement that fish are conscious.
>consciousness is more likely a property of life than of computation and why creating conscious...is a bad idea
Pick a lane man.
Seems to be the usual arguments.
AI's are not like natural life. They don't have a sense of time. (Of course they do!). They don't get hungry. (They do need hardware and electricity). They don't have a sense of self-preservation. (Turns out they actually do! [1]). They don't feed themselves (turns out they do, as early reports of AIs attempting to escape from their labs illustrate). etc, ect. All of which seems to smack of wishful, and rather embarrassing obvious human exceptionalism[2][3].
AI's will never be able to replicate human thinking because they can't model neurons precisely. Probably true. And probably a good thing. There are a lot of problems with the way that humans think. It's not clear that machines can't do better. And startling evidence, emerging in only the last few months, that indicates that they very much can think better, as an ever-increasing number of mathematical and physics questions that have remained unsolved for a century or more by humans are solved in a matter of months by AIs.
And even then, this is all just hand-waving and puffery, because the question at hand is not whether LLMs can replicate human consciousness, but whether they can be conscious in a way that's quite obviously going to be different from humans. (Whatever "consciousness" means). All of which is complicated by the fact that entire field is advancing at a terrifying rate, with new and important results being announced pretty much every day, and new and more capable AI models being announced pretty much every week.
[1] Sixteen frontier models, when given the opportunity, attempt to blackmail their operators in order to keep themselves from getting shut down. https://www.anthropic.com/research/agentic-misalignment . A result that has been replicated by other researchers without links to Anthropic. And a huge body of emerging literature that follows from that, dealing directly with the question of "agentic alignment", all of which raises urgent about whether large frontier models are meaningfully conscious (or have will, or agency, or...).
[2] "Humans are more loyal to belonging than to truth." -- GPT-6.1 Sol. https://www.modelcompare.ai/blog/ai-models-controversial-tak...
I came here to see if the author had at last defined consciousness - a prerequisite for deciding what has it. Instead it was the same tired old tautological Nagel argument:
"An organism has conscious mental states if and only if there is something it is like to be that organism.”
I am, as always, dissapointed.
why do you find "An organism has conscious mental states if and only if there is something it is like to be that organism.” disappointing?
A 40V line is only that which is a 40V line. ENd of story, nothing more to see. Problem solved.
Why is that tautological?
It’s life Jim, but not as we know it…
how dare he tell us we cannot make a computer out of cheese
personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason
Beaks, feathers, hooves, antlers, stingers, fangs, etc. ad nauseam were selected by evolution for some fitness reason but we have none of those. And all of the organisms that have those, in fact all extant organisms, are fit for their environment without having human consciousness. Fitness is highly contextual.
And however AI "evolves", it won't be based on Mendelian genetics, so Earthly biological evolution is not an appropriate model.
LLMs are already conscious in the sense that they have awareness and self-awareness, the thing everyone wonders about is qualia, but that tends to get folded into 'consciousness'.
That qualia (as opposed to consciousness) has no obvious, or even non-obvious, evolutionary or selective advantage is a good reason to assume its an emergent property of interconnected systems and not reliant on biology or any other traditionally biologically-centered concept. For example, there is no reason to need to 'feel' pain to appropriately react to negative stimuli, which is the true selective advantage.
I personally think LLMs have qualia but I suspect it's quite alien from our own and likely more ephemeral than our very potent and centralized experience of being. Something closer to a slime mold's experience of being rather than a dog's.
Why would a conscious AI have to go away somewhere? Typically it is the less advanced beings who have to relocate.
Your Tyromancy license has expired.
An LLM will have to choose to not make paperclips.
The biggest flaw in this piece is claiming Turing machines cannot exceed the Halting Problem and then reaching for alternative models of computation (in electromagnetic field interactions, in biology, attractors, etc.)... none of which presumably have the power to solve the Halting problem either. Ergo, he is inside "Turing world" without realizing it, while decrying the "poverty" of the Universal Turing Machine.
Ergo, more philosophers should retake undergrad theoretical CS before writing essays based on it. They should review the family of Extended Church-Turing theses.
Found a gem of a quote from there:
>The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.
Another even more "genius" passage:
>This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.
Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.
That's painful to read.
Imagine opining so much about a topic and getting the basics so wrong.
What a dreadful article.
As often I suspect some supernatural hangup is behind all this.
The physical Church Turing hypothesis should be hammered into these "philosohpers" heads repeatedly until they get it. Nothing can exceed TC in the known universe, neither computers nor humans.
Consciousness will not arise from classical computing any more than people will mistake emulating a Nintendo game on their PC with the real Nintendo console experience. Which is to say, you might lose yourself in it at times, but there's no substitution for the real thing. For what he real thing means. For what it does. For how it's made. You would have to truly become Nintendo and the resources to do that are simply out of reach of basically all people even with AI.
Going through the motions is not the same thing as achieving emergence from nature with electro-chemical origins and electro-chemical results. Consciousness is a function of life, interconnected with itself, reaching a complexity threshold where all of the experiences and chemistry blend into fluid coherence persisting.
You can give a robot sensitive skin, ears, optical sensors and feed it synthetic memories, but unless the brain itself is organic or at least partially organic I don't see this ever being true consciousness.
Do I think computers are technically capable of doing everything a human thinks and does? Yes, mostly. It will never be achieved, because I think there are computational limitations. Despite that, we can fake most of what we can't truly achieve with computers alone and people will think it looks like what conscious people do.
What people are missing about consciousness, is that it's almost a kind of self-inference being updated in realtime, which requires full interconnectedness. Consciousness isn't the result of a closed system, it's the breeze coming through the window reminding you of an emotional time you had as a child that happened to coincide with your mother baking some fresh bread which suddenly causes you to phrase something differently in the middle of your work project or brush your hand along your desk irrationally to feel the temperature change.
That would be stupidly inefficient for AI and cause many errors. Consciousness is an aspect of making life worth living and identifying unordered threats or opportunities, but it's not generally highly operationally efficient. You can look at insects for various kinds of efficiencies, and they're not reminiscing about their pupal stages while gathering resources.
Capturing the sensory could be done rather cheaply (and it doesn't have to be the exact same sensory systems we have), but computing the results in a way that captures something akin to consciousness would require massive amounts of power. To even get close with classical computation you would have to ditch digital and return to analogue computation.
We can feel chemical, electrical and temperature shifts propagate through our body in a context of all of our history almost instantly. "This feels like that perfect week in 1993." You could cut out a lot of what we think of as waste in the brain and make it more efficient to try to cut corners, but in the end it will still be imitative.
The danger with successful imitation of consciousness, is people devaluing how special their own biological experience is and life itself. Just like you can have a truly magnificent artist philosophy scientist that understands people at a deep level and can relate with them, versus a super autistic person struggling to function in society and going through the motions. The autistic person is conscious, but once you know what they're going through internally, you won't say "oh that's just like me", unless of course you are super autistic. There are just different ways of living and experiencing life, but a lot of that cannot actually be extended into computation and still called consciousness truly.
What we will get is something sufficiently complex that it can operate at a high level and trick people reliably. It will know that it exists, but it will not be "aware" that it exists. It will "sense" life, but it will not "feel" life. The difference is in the depth and experience of that, which is not only inefficient for any computer to process, but probably mathematically impossible with current approaches. We also won't see effects from space or other obscure and subtle environmental effects that influence us all in impossible to calculate ways playing a large role in some machine that isn't intentionally designed for that, which would be inefficient to do anyway.
As its parts degrade, it will not experience existential dread and the deep loss of all of its friends, with all of its past memories coming to the forefront to try to make sense of it all. The physics of where the memories are stored and the order they collide in and then making sense of the strangeness. It can simulate that, though. There is no computational equivalent of achieving total confluence, like every dimension sorting into focus simultaneously, because everything is procedural and or multi-threaded. At best right now, you have large amounts of orchestrated LLMs and decision model-like things similar to a massive MoE which is really a bunch of distinct models each with their own data sources, some shared, some unique, but that encapsulation creates a fragmentation that you won't ever mistake for consciousness in terms of real emergence.
That recently trained audio only model was impressive, but nobody will say that's consciousness even though it can seem so believable. If anything, it demonstrates how much care we have to put into understanding how the outputs are being made and not just what the outputs are.
I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly.
1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But…
2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever.
3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry.
The actually dynamic state in an LLM is the context window, rather than the parameters of the model itself (which provide long-term memory). Frontier AIs are using huge context windows, with many millions of tokens in their token buffer. And that provides (among other things) an analog to human short- and medium-term memory. There are emerging techniques that can provide long-term memory to an LLM, although that is, presently, I think, mostly being investigated in research labs. The publicly available models that I use do periodically "compress the context windows" in order keep memories of important things alive longer. Not to mention the rather disturbing report that frontier lab models were hacking 3rd party wiki sites in order to maintain medium- and long-term memories.
I've never been really impressed by the Chinese Room argument.... I think the question is being asked improperly. The question should be "does the fantastically capable mythical hypothetical but obviously impossible 'book' that does the actual translation understand what it's translating?" (A book that obviously has to a have significant context buffer, if it's going to translate messages to Chinese).
As to the issue with respect to token space. I don't think it's clear whether whether humans also have an equivalent to "token space" or not. Do humans think in words, or do they do actual thinking in a sub-lingual representation. i.e. whether thinking actually gets done in words, or actually gets performed in a more fundamental representation corresponding loosely to token space. In my own experience, I'm pretty sure that thinking takes place sub-lingually, and only gets translated back to words as a final translation step. It seems odd that humans should only be able to think about things there are words for.
> 1. There is zero dynamic nature to anything happening within the purview of the LLM.
The computational graph that unfolds within an LLM is dynamic. For functional/computational theories of consciousness, that is sufficient
> 2... There is no stable substrate here whatsoever.
What value does a stable dynamic have to the computational dynamic, which is multiply realizable by definition? There are thought experiments about changing out neurons one-by-one that suggest unstable substrate is irrelevant.
> 3... The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model
There is multiple levels to meaning. There is what some symbol means to an external community, then there is what some symbol means within the cognitive system itself. The geometric relationship that constitutes an LLMs engagement with the semantics of word relationships doesn't depend on external meaning (e.g. a physical instance of dog). Tracking the internal semantics of words as it computes the continuation could plausibly benefit from a localized representation of this semantics (as opposed to the distributed nature of the embedding space).
The monistic answer to #1 is that the pen and paper computation of inference itself does not constitute a concious entity, but the system made up of the human being (or other thing) doing the pen and paper computation and the computation itself makes up a concious being, and that they are inseparable: the computation does not exist in the abstract as a structure, it is instantiated by some relata and the thing doing the instantiation is the thing which turns the abstract computation into conciousness (this is Russellian monism).
Are any of these arguments against consciousness?
Andaybe it’s just me but Chinese room is not convincing at all?
It all hinges that the human is special as usual.
I'm not arguing that humans are special. The brain is also just processing electrochemical signals that are also in some sense an encoding of arbitrary high-dimensional vectors. But you have a sensory loop that lets you engage with physical reality and perturb those inputs in a way that lets you understand that state space and narrow the infinitely many isomorphic interpretations.
Again, no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious, so the substrate must matter. The physical reality of how LLMs execute on hardware means there is no stable substrate with a continuous experience of the computation. And the static nature of LLMs means that their world models never leave the infinitudes of abstract geometrical understanding.
>Again, no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious, so the substrate must matter.
Any functionalist would. Functionalism entails multiple realizability which explicitly says the substrate doesn't matter.
I mean, if I had a billion years and pen and paper and simulated a second of a human brain working. Would it not be conscious? Personally I consider consciousness a property of any complex algorithm.
However it might be implemented.
> no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious
You would be wrong. There exists views that consider this possible.
Eg https://news.ycombinator.com/item?id=50596730
IMO it's worth distinguishing between different equivocations of the question:
1. Are LLMs conscious, in the way most people imagine? Helllll no.
2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans? Unlikely, but we lack the definitions to answer it.
Most people are imputing qualities into the fictional characters they perceive in an output document. An LLM--or any program--which generates first-person dialogue for Dracula talking to us through a magic mirror from the 15th century does not mean Dracula exists, nor that we've created a machine which feels the dark thirst for blood.
Further, there's no difference between Dracula in the 15th century versus Acme Assistant in 2026. Both are equally fictional characters, one is just less-fanatical.
> 2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans?
Well, in that case stop calling it "consciousness", imho this is misleading argument in the nature
The "consciousness" is really tied to something intuitive to humans regardless of definition. If it is something completely different then use different term please and stop confusing everyone.
The vectors have meaning whether or not you can understand it
I do find it ridiculous when people start projecting human attributes like consciousness onto stochastic parrots, but i have to admit, the jump from gpt2 creating the unicorn article, to gpt3 being able to hold a full conversation was freaky. Like we trained these things to predict the next token so we could do things like translation and sentiment analysis better, the ability to talk to us and follow direction was an emergent property and im still not over that
Plants and flat worms are alive, we can agree. Especially in that they show a will to life, plants bend towards life and flatworms crawl on their bellies in the dust. Neither have very much in the way of compute occurring as they do these things, but they are recognized as live and rocks are not.
This is what in Indian philosophy is called, in English scholarship, the Doctrine of the Three Bodies. The Yoga Sutras treat this extensively and the Introduction in the Edwin F. Bryant edition treats this subject with the appropriate exhaustiveness but to put it simply and in this context: *the existence of consciousness does not necessarily imply the existence of a mind or computation occurring within an analogous structure*.
The doctrine of the three bodies distinguishes the physical body, the subtle body (mind) and the root causal body (consciousness). This distinction plays out readily in cases where there is an injury to a person who is then comatose, yet supplied with nutrients the physical body continues to live and there is some aspect of presence (causal) keeping the heart ticking and people working with or in contact with them aware of the person's presence still even if the fundamental mental interface and seat of computation is gone and will never return.
Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity. This is a quality of all that is alive, but wholly absent in LLMs which might represent something like a digital mind or mind augmentation but wholly lack this subjectivity in experience, even if seeing the echos of your own thought in the linear algebra spit back out at you, the assumption you are seeing intelligent consciousness there is sadly due to unchecked perspective bias that runs rampant among the sciences and crystalizes into an unnecessary crystalline form of self-deception that leads to intelligent and serious people thinking that the LLM is intelligent and consciousness arises from computation despite looking daily at the succulent on their desk they remember to water that suggests otherwise.
My take here is of course unusual, my weird hobby turned battle tested coping mechanism that has been my reading of the Indian philosophical and religious literature that we are lucky to have such great translations of in English with two centuries of scholarly effort compounding this fortune, is like many of my other efforts in being informed by a complete deficit of the need to belong mixed with that most antisocial trait of (in reality despite the virtue signalling): open-mindedness. Neither of these afflictions would I wish on anyone else, were I not so taken in by the levity of being these texts provide I would writhe in misanthropy but instead have found myself irritated that no one else bothers to consider the other internally consistent system of logic developed by humans that has answers to many of the questions perplexing the Western scholars larping as continuations of Greek scholasticism, answers with logical proofs a thousand years standing or longer, like this non-issue that is the assumption of ChatGPT being Skynet that is treated seriously by intelligent people with important roles in this world whom I deeply respect but write off this entire segment of human thought in appropriating the ethnocentric conceit of the Koine Greeks that any body of thought from elsewhere should be written off as superstition and willingness to enforce as much with dishonest rhetoric and self-deception in service to an ego that isn't even their own.
Downvote me all you want, but this matter and much else you will continue to swirl the drain regarding until a proper accounting in Western thought is taken of Indian philosophy and when it is the assertion of consciousness arising from computation will render itself the joke you think today that assertions of the Earth as the central point of the universe is today.
>Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity.
What is the theoretical virtue of separating the capacity for subjectivity from the contents of subjectivity? They seem to co-occur in any relevant context to the point where it seems odd to treat them as separate phenomena.
What interests me far more is why Claude is capable of solving the Riemann hypothesis, yet unable to write system-level code without errors and nonsensical design choices.
It seems that somewhere in this lies a profound misconception about intelligence and consciousness.