It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
A bee is an autonomous walking, climbing, and flying drone that investigates its environment, collects resources, builds structures, and coordinates with other drones.
We're totally incapable of building an AI that can do anything resembling that. We're still at the phase where robots walking on rough terrain without falling over remains a bit impressive.
I doubt the limitation is that we can't produce enough raw compute to replace a single bee.
I’m in agreement with you in that raw compute isn’t really the only missing piece, but I’m not in agreement we have enough compute to fully simulate even simple insect brains
I feel like we are talking past one another here. I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea.
Now if you meant we don't know enough about bees to actually model what its brain does and create a processor to actually process an artificial bee brain, then yes, we don't have that ability yet.
----
Aside, when I watched Netflix show black mirror such as the grain episode, I always kept getting stuck at how are they lowering this tiny device... What kind of battery technology works here was my question even though this is science fiction.
> I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea.
A bee weighs 100mg. If they are 5% brain, their brain weighs 5mg. 5mg of carbon is 2.5 * 10^20 atoms.
The largest supercomputers, at hundreds of trillions of transistors, come in at 2-5 * 10^14 transistors.
I don't think that combining all the computers in the world would give us 1 transistor per atom in a bee brain, and I additionally have to imagine transistors are incapable of simulating an atom by multiple orders of magnitude (i.e. 1 atom would require > 1 * 1 ^ 4 transistors) in realtime.
So, I would argue that our level of compute is still insufficient to simulate even a simple insect brain in realtime. Perhaps you could compute 1 second of bee thinking in an hour using all our compute.
And then, of course, comes into play that we have no idea how to simulate an atom in full fidelity.
> A bee weighs 100mg. If they are 5% brain, their brain weighs 5mg. 5mg of carbon is 2.5 * 10^20 atoms.
Any rational characterization of the problem has left the building here.
Nobody wants to simulate any creature's brain or nervous system atom for atom.
Most of what atoms do in any cell, including neurological cells, is operate a vast number of complex but common survival systems. And the very small fraction supporting specifically neurological behavior, do not meaningfully contribute at the level where specifics of individual atoms matter, but at a level many orders of magnitude higher in scale.
We’re talking about whether or not humanity has enough compute to simulate a bee brain, how can us having orders of magnitude less transistors than there are atoms in the system we wish to simulate possibly be hand waved away? A single transistor doesn’t do much, regardless of how much you wish to baldly assert atoms are supposedly trivial.
Perhaps you are meaning to compare transistors with neurons? If we do that, then yes, a transistor is a lot simpler than a neuron. But transistors are also insanely fast compared to neurons, so transistors do far FAR more, per unit of time.
A bee brain might have a million neurons, operating at maximum speed of about 250 Hz.
10^6 neurons x 250 Hz
= 2.5 x 10^8 Hz for a bee brain.
We can easily model that many neurons with a trillion transistors, operating at 100 Ghz. (If that sounds fast to you, keep in mind that CPU clock speeds account for many transistors switching in series.)
10^12 transistors x 10^11 Hz
= 10^23 Hz for a CPU
That is a factor of 4 x 10^14 more powerful.
So yeah, the only problem for modeling a bee brain is identifying the organization of its neurons.
Nature does a lot with a little. Trillions of bee life years went into designing a really efficient bee brain.
---
Atoms:
A bee brain might have 10^20 atoms. Atoms interact at speeds that are far beyond anything we are talking about here. But they don't "switch", they bounce around every which way and slowly end up reacting when they connect with the right conditions. This is called Brownian motion and its not computation, its natures way of using the chaos produced by heat to give compounds so many chances to find their right context that they eventually do.
While we could not easily model that many atoms, nobody wants to (with regard to bees). Atoms are neither the "transistors" or the neurons of a bee brain.
Oh yeah, you're right, we have 4 * 10^14 more compute than an insectoid brain in real time. Come the fuck off it.
This is the same guy who said rational characterizations had left right before waving away the complexity of a trillion atoms. Painful to imagine how smarmy you are.
Well you have left your written record of (1) persistently confusing atoms with neurons, (2) an inability to explain where your confusion comes from outside of just repeating it, along side (3) snide responses to someone who in good faith took the time to explain the difference to you.
One more time:
Not understanding something is not the same as needing high compute. And the amount of compute we have today is colossal.
Simulating a high level system (neurons or transistors) is not the same as simulating their low level implementations (all the atoms and electrons in either of them).
Some large LLMs (interactive summaries of a large percentage of the entire human race's verbalized knowledge at moderate depth), can be run on a single MacStudio M3 Ultra with 512GB of RAM.
Modeling a bee brain will require a tiny fraction of that computationally.
> Most of what atoms do in any cell, including neurological cells, is operate a vast number of complex but common survival systems. And the very small fraction supporting specifically neurological behavior, do not meaningfully contribute at the level where specifics of individual atoms matter, but at a level many orders of magnitude higher in scale.
I don’t see where I reference neurons, care to quote me rather than adding misleading citation looking (1) strings to your text?
Indeed, looking towards full compute simulation of a physical system, I reference one of its smallest physical components - atoms - and draw an analogy to the transistor in compute. You make no argument for why an atom needn’t be simulated in a system but nonetheless hand wave it away, lifting yourself to neurons as if their behavior is well understood, finally acting as if clock speed of a transistor is somehow related to the unsourced “clock speed” of a neuron.
So, to me it seems pretty transparently obvious you’ve failed to grasp the two topics in play here - whether there is enough brute force compute available, and what models of simulating a system like a brain exist - and in your confusion, went ahead and self aggrandized your own typing to cast out the conversation that played out before you joined.
I encourage you to not respond, and instead to use that energy to read someone else’s argument from a steel man perspective rather than this diminishing arrogance you display.
You’re not trying to simulate a brain atom for atom. Individual atoms don’t do much on their own. Even if you did do that, you’d really need to simulate electron flow, which is whole other level.
Now, simulating what outputs the bees brain yields from a set of stimuli — that could be done. If it could be done as fast as a brain is a whole other question, and I’m not sure of that answer.
I think the problem here is the physical hardware that will navigate and collect information from this environment? In this case, a biological robot makes more sense than a mechanical/electronic one. If you go down that route, though, the best AI will be a human brain. We have been training and selecting these for quite a while now.
On the other hand, a large part of the complexity of human hardware randomly evolved for survival and only recently started playing around in the higher-order intellect game. It could be that we don't need so many neurons just for playing intellectual games in an environment with no natural selection pressure.
Evolution is winning because it's operating at a much lower scale than we are and needs less energy to achieve anything. Coincidentally, our own progress has also been tied to the rate of shrinking of our toys.
Evolution has won so far because it had a four billion year head start. In two hundred years, technology has gone from "this multi-ton machine can do arithmetic operations on large numbers several times faster than a person" to "this box produces a convincing facsimile of human conversation, but it only emulates a trillion neurons and they're not nearly as sophisticated as real ones."
I do think we probably need a new hardware approach to get to the human level, but it does seem like it will happen in a relative blink of an eye compared to how long the brain took.
But we don't even need a human brain. We already have those, they take months to grow, take forever to train, and are forever distracted. Our logic-based processes will keep getting smaller and less power hungry as we figure out how to implement them at even lower scales, and eventually we'll be able to solve problems with the same building blocks as evolution but in intelligent ways, of which LLMs will likely only play a minuscule part of the larger algorithms.
I think current LLMs are trying to poorly emulate several distinct systems.
They're not that great at knowledge (and we're currently wasting most of the neurons on memorizing common crawl, which... have you looked at common crawl?)
They're not that great at determinism (a good solution here is that the LLM writes 10 lines of Python, which then feed back into the LLM. Then the task completes 100% of the time, and much cheaper too).
They're not that great at complex rules (surprisingly good actually, but expensive and flakey). Often we are trying to simulate what are basically 50 lines of Prolog with a trillion params and 50KB of vague English prompts.
I think if we figure out what we're actually trying to do with these things, then we can actually do each of those things properly, and the whole thing is going to work a lot better.
>But we don't even need a human brain. We already have those, they take months to grow, take forever to train
This is a weird argument considering LLMs are composed of the output of countless hours of human brains. That makes LLMs, by definition, logarithmically worse at learning.
Not all artificial neurons are LLMs. Machine learning can be applied to any kind of large data set, not just human prose, and will start finding useful patterns before a human brain has time to learn how many fingers it has.
We use logic to design our technology, but evolution does it by literally shaking all the atoms into place, no design involved. Our brains were created randomly.
It appears logical, but it falls under the same threshold. We say it has function, which is logical, but its used for something we could or may not have planned for. That's evolution. The logic is a pretext.
To be fair to the raw capabilities of the semiconductor industry, a 100mm^2 die at 3nm can contain on the order of 1~10 trillion features. I don't know that we are actually that far off in terms of scale. How to arrange these features seems to be the difficult part.
The EDA [0] problem is immune to the bitter lesson. There are certainly specific arrangements of matter that can solve this problem better than a GPU/TPU/CPU can today.
The bigger issue though is that we can't scale that die to the approximate volume of a human brain in any dimension.
Those feature sizes are tiny, yes, but we struggle to put them in a block the size of a human brain and keep it cool enough to be useful (or even make it affordable).
This is a great summary! I've joked with a coworker that while our capabilities can sometimes pale in comparison (such as dealing with massively high-dimensional data), at least we can run on just a few sandwiches per day.
> To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs
This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
Edit: As for the rest of your argument, it's not so clear cut. An LLM can produce a complete essay in a fraction of the time it would take a human. So yes, a human brain only consumes about 20W but it might take a week to produce the same essay that the LLM can produce in a few seconds.
Also, LLMs can process multiple prompts in parallel and share resources across those prompts, so again, the energy use is not directly comparable in the way you've portrayed.
> This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
I think it's more than just scaling, you need to understand the functional details to reproduce those functions (assuming those functions are valuable for the end result as opposed to just the way it had to be done given the medium).
An interesting example of this neuron complexity that was published recently:
As rats/mice (can't remember which) are exposed to new stimuli, the axon terminals of a single neuron do not all transmit a signal when there is an action potential, they transmit in a changing pattern after each action potential and ultimately settle into a more consistent pattern of some transmitting and some not.
IMHO: There is interesting mathematical modeling and transformations going on in the brain that is the secret sauce for our intelligence and it is yet to be figured out. It's not just scaling of LLM's, it's finding the right functions.
Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily. I was making a more general point that a single-I/O function can pretty trivially replicate a multi-I/O function, so the OP's point that "LLM neurons" are single-I/O and bio neurons are multi-I/O doesn't mean much. Estimates of brain complexity have already factored this in, which is why we know we're still a few orders of magnitude away from the number of parameters needed for a human brain in a raw compute sense.
However, the human brain has extra parameters that a pure/distilled general intelligence may not actually need, eg. emotions, some types of perception, balance, and modulation of various biological processes. It's not clear how many of the parameters of the human brain these take up, so maybe we're not as far as we think.
And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical. I think general intelligence will likely have multiple models which achieve similar results, just like there are multiple ways to sort a set of numbers.
> Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily.
Ya, I realized later that the LLM scaling part of my post sounded like it misinterpreted what you said when it was really a separate point unrelated to the topic of neurons that just happened to include the word "scaling" also.
I do agree with you somewhat that just because biological neurons are vastly more complex and functional than typical artificial neurons, it just means we need more artificial neurons to achieve similar functionality.
> Estimates of brain complexity have already factored this in
I don't agree with this, the estimates I've seen don't seem to factor it in, and many of those estimates were prior to things discovered within just the last 5 years that expose significantly more complexity and capability that needs to be understood first.
> And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical.
I kept reading that people wanted to use spiking networks and I thought the same thing as you, it didn't seem to provide a benefit. A while ago I read some paper about why they want to use spiking networks and I can't remember the details but they described some functional capabilities that really were much easier with spiking. I vaguely remember that it had to do with processing real-time sensory information, it was easier to synchronize signals based on frequency instead of trying to rely on precise single signal timing (something like that). And I think there were benefits in other areas also.
I agree with both of you, but scaling isn't feasible with this paradigm. You could need continent-sized hardware to approximate general intelligence with the current paradigm.
> You could need continent-sized hardware to approximate general intelligence with the current paradigm.
I doubt it, if by "current paradigm" you mean the hardware and general execution model, eg. matrix math. Model improvements from progress in algorithms have been outpacing performance improvements from hardware progress for decades. Even if hardware development stopped today, models will continue improving exponentially.
It's not even that. The architecture(s) behind LLMs are nowhere near close that of a brain. The brain has multiple entry-points for different signals and uses different signaling across different parts. A brain of a rodent is much more complex than LLMs are.
LLM 'neurons' are not single input/single output functions. Most 'neurons' are Mat-Vec computations that combine the products of dozens or hundreds of prior weights.
In our lane the only important question to ask is, "Of what value are the tokens these models output?" not "How closely can we emulate an organic bran?"
Regarding the article, I disagree with the thesis that AGI research is a waste. AGI is the moonshot goal. It's what motivated the fairly expensive experiment that produced the GPT models, and we can look at all sorts of other hairbrained goals that ended up making revolutionary changes.
"To simplify it greatly, an LLM neuron is a single input single output function". This is very wrong unless I'm mistaken. A synthetic neuron is multiple input single output.
Ten thousands of extremely complex analog inputs, one output with several thousand of targets that MIGHT receive the output with different timing and quality.
One neuron is ufathomably complex. It‘s offensive to biology to call a cell in a mathematical matrix neuron.
b) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.
“And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.”
I’ve always thought about nature didn’t evolve to use electricity as its primary means of energy. Instead it uses chemistry. It’s quite curious, really.
Like a tiny insect is chemistry powered. It doesn’t need to recharge batteries, it needs to eat and breathe oxygen.
What if our computers started to use biology and chemistry as their primary energy source?
Or will it be the case that in the end using electricity as the primary energy source is more efficient for “human brain scale computation”, it’s just that nature didn’t evolve that way…
"Wetware" as it were , i remeber some research article some time back where they grew some kind of 'brainlets' and had them functional as far as memory was concerned[1].
Would be a interesting to see how that tech would progress while incororatd in the current silicon/photonic devices for input/output.
Downside would be the durability of the organic matter and replacement.
>Downside would be the durability of the organic matter and replacement.
thinking about the consequences of consumer grade "organic computing" gets weird really fast. How do you interface biological matter with peripherals? What about toxicity? What about pathogens? Not only as targets, but as vectors too. What about senescence? Would my computer catch a cold or get Alzheimer's? What about energy? Would I have to buy proteins/sugar for my PC? Would a "beefy PC master race" kind of machine big enough to gain sentience? Would my PC need to literally sleep?!
I always thought optical signalling would be a interesting i/o pathway being non intrusive in HumanComputer interaction - rather than electrical signalling probes ala neuralink.
Yeah and sentience open a whole new can of ethical/moral implications to sort through.
Med resident here: AFAIK the 80-90 billion neuron is misleading: more than 80% of them are in the cerebellum and are mostly a low pass filter for motor signals. People born with no cerebellum are of normal intelligence. And we don't know how much of the neocortex is actually useful for consciousness but apparently a minority of it.
Minor correction here. You are correct about hardware being an issue, but the magnitude is much greater. You have a lot more than "thousands" of inputs. In the hand alone you have ~40,000+ tactile corpuscles (sensing regions). And that's just one mode. The eye has ~7 million cones and 80 million rods. There is processing and quantization performed by each of those cells and each of the additional cells those signal, throughout the entire sensory-brain system. The amount of data the human brain processes is many orders of magnitude greater than even our largest exascale computers. We are at least 3 decades from AGI if we need equivalent data processing as the human brain, and that's optimistic.
Like you mention, each individual neuron or synapse includes fully parallel processing capability. With signals conveyed by dozens of different molecules. Each neuron (~86 billion) holds state information in addition to processing. The same is true for each synapse (~600 quadrillion). That is how many ~10 Hz "cores" the human computational system has.
The hubris of the AI community is laughable considering the biological complexity of the human body and brain. If we need anywhere close to the same processing capability, there is no doubt we are multiple massive hardware advances away from AGI.
I agree with this up till saying we must be very far from AGI. I don't think we're close, but the scale of human inputs doesn't tell us anything about it. A useful AGI need not be capable of human level cognition, and human level cognition need not require the entire human biological or nervous systems - we're a product of millions of years of undirected random evolution, optimized to run a fleshy body and survive African plains predators. This whole thing we do of thinking and science and engineering is a quirk that made us very adaptable, but how much of what we are is required to implement it isn't clear (i.e. a human minus a hand can still understand advanced mathematics, there are blind programmers etc.)
Sure, but there are animals with much larger brains on Earth which have - we believe - a reasonably high level of cognition but have not achieved the technological and engineering feats which we have (i.e. dolphins, whales, elephants as naive examples based on brain mass and complexity).
Conversely you have birds - with much smaller brains - which also don't achieve those things but display advanced language skills, have apparent societal structure and despite our inability to understand it seem to have enough language to communicate advanced concepts.
You are absolutely correct. There are multiple algorithmic advances required in addition to hardware advances.
Parrots have hundreds of millions to a few billion neurons, which are just as much parallel-local state-proceasing units as the ones in the mammalian brain. We haven't done a great job of simulating a ~300 neuron c. elegans worm. Not that simulation is required for equivalent intelligence. I'm just saying these analog machines are much much more complex and powerful than the average AI aficionado gives credit. So complex that we are nowhere near AGI.
I wrote a concrete expected‑value model for AGI that anchors rewards in the 15–30T USD Western white‑collar payroll, adds spillovers on 60T GDP, includes transition costs, and varies probability explicitly. Three scenarios (optimistic, mid, pessimistic) show when the bet is rational versus value‑destroying—no mysticism, just plug‑and‑play numbers. If you’re debating AGI’s payoff, benchmark it against actual payroll and GDP, not vibes.
A lot of people say that, but no one, not a single person has ever pointed out a fundamental limitation that would prevent an LLM from going all the way.
We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI.
Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic and physical constraints on continuing this trend indefinitely.
OOD is still unsolved problem, they basically struggle under domain shifts and long tail cases or when you try systematically new combinations of concepts (especially on reasoning heavy tasks). This is now a well documented limitation of LLMs/multimodal LLMs.
Work on COT faithfulness shows that the step by step reasoning they print doesn't match their actual internal computation, they frequently generate plausible but misleading explanations of their own answers (lookup anthropic paper). That means they lack self knowledge about how/why they got a result. I doubt you can get AGI without that.
None of this proves that no LLM based architecture could ever reach AGI. But it directly contradicts the idea that we haven't found any limits. We've already found multiple major limitations of the current LLMs, and there's no evidence that blindly scaling this recipe is enough to cross from very capable assistant to AGI.
A lot of those failings (i.e. COT faithfulness) are straight up human failure modes.
LLMs failing the same way as humans do on the same tasks as humans is a weak sign of "this tech is AGI capable", in my eyes. Because it hints that LLMs are angling to do the same things human mind does, and in similar enough ways to share the failure modes. And human mind is the one architecture we know to support general intelligence.
Anthropic has a more recent paper on introspection in LLMs, by the way. With numerous findings. The main takeaway is: existing LLMs have introspection capabilities - weak, limited and unreliable, but present nonetheless. It's a bit weird, given that we never trained them for that.
This is all nonsense and you are just falling for marketing that you want to be true.
The whole space is largely marketing at this point, intentionally conflating all these philosophical terms because we don't want to face the ugly reality that LLMs are a dead end to "AGI".
Not to mention, it is not on those who don't believe in Santa Clause to prove that Santa Clause doesn't exist. It is on those who believe in Santa Clause to show how AGI can possibly emerge from next token prediction.
I would question if you even use the models much really because I thought this in 2023 but I just can't imagine how anyone who uses the models all the time can possibly think we are on the path to AGI with LLMs in 2025.
It is almost like the idea of a thinking being emerging from text was a dumb idea to start with.
Which is: flesh apes want to feel unique and special! And "intelligence" must be what makes them so unique and special! So they deny "intelligence" in anything that's not a fellow flesh ape!
If an AI can't talk like a human, then it must be the talking that makes the human intelligence special! But if the AI can talk, then talking was never important for intelligence in the first place! Repeat for everything.
I use LLMs a lot, and the improvements in the last few years are vast. OpenAI's entire personality tuning team should be loaded into a rocket and fired off into the sun, but that's a separate issue from raw AI capabilities, which keep improving steadily and with no end in sight.
Breaking down in -30C temperatures is also human failure mode, but doesen't make cars human. They both exhibit the exact same behavior (not moving), but are fundamentally different
Both rely on a certain metabolic process to be able to move. Both function in a narrow temperature range, and fail outside it. Both have a homeostatic process that attempts to keep them in that temperature range. Both rely on chemical energy, oxidizing stored hydrocarbons to extract power from them, and both take in O2-rich air, and emit air enriched in CO2 and water vapor.
So, yes, the cars aren't humans. But they sure implement quite a few of the same things as humans do - despite being made out of very different parts.
LLMs of today? They implement abstract thinking the same way cars implement aerobic metabolism. A nonhuman implementation, but one that does a great many of the same things.
LLMs are bounded by the same bounds computers are. They run on computers so a prime example of a limitation is Rices theorem. Any ‘ai’ that writes code is unable (just like humans) to determine if the output is or is not error free.
This means a multi agent workflow without human that writes code may or may not be error free.
LLMs are also bounded by runtime complexity. Could an llm find the shortest Hamiltionian path between two cities in non polynomial time?
LLMs are bounded by in model context:
Could an llm create and use a new language with no context in its model?
Assuming you want to define the goal, "AGI", as something functionally equivalent to part (or all) of the human brain, there are two broad approaches to implement that.
1) Try to build a neuron-level brain simulator - something that is a far distant possibility, not because of compute, but because we don't have a clear enough idea of how the brain is wired, how neurons work, and what level of fidelity is needed to capture all the aspects of neuron dynamics that are functionally relevant rather than just part of a wetware realization
OR
2) Analyze what the brain is doing, to extent possible given our current incomplete knowledge, and/or reduce the definition of "AGI" to a functional level, then design a functional architecture/implementation, rather than neuron level one, to implement it
The compute demands of these two approaches are massively different. It's like the difference between an electronic circuit simulator that works at gate level vs one that works at functional level.
For time being we have no choice other than following the functional approach, since we just don't know enough to build an accurate brain simulator even if that was for some reason to be seen as the preferred approach.
The power efficiency of a brain vs a gigawatt systolic array is certainly dramatic, and it would be great for the planet to close that gap, but it seems we first need to build a working "AGI" or artificial brain (however you want it define the goal) before we optimize it. Research and iteration requires a flexible platform like GPUs. Maybe when we figure it out we can use more of a dataflow brain-like approach to reduce power usage.
OTOH, look at the difference between a single user MOE LLM, and one running in a datacenter simultaneously processing multiple inputs. In the single-user case we conceptualize the MOE as saving FLOPs/power by only having one "expert" active at a time, but in the multi-user case all experts are active all the time handling tokens from different users. The potential of a dataflow approach to save power may be similar, with all parts of the model active at the same time when handling a datacenter load, so a custom hardware realization may not be needed/relevant for power efficiency.
You could do an architectural search, and Google previously did that for CNNs with it's NASNet (Network Architectural Search) series of architectures, but the problem is you first need to decide what are the architectural components you want your search process to operate over, so you are baking in a lot of assumptions from the start and massively reducing the search space (because this is necessary to be computationally viable).
A search or evolutionary process would also need an AGI-evaluator to guide the search, and this evaluator would then determine the characteristics of the solution found, so it rather smacks of benchmark gaming rather than the preferred approach of designing for generic capabilities rather than specific evaluations.
I wouldn't say we don't know how LLMs "work" - clearly we know how the transformer itself works, and it was designed intentionally with certain approach in mind - we just don't know all the details of what representations it has learnt from the data. I also wouldn't say LLMs/transformers represent a bitter lesson approach since the architecture is so specific - there is a lot of assumptions baked into it.
Hard problem of consciousness seems way harder to wolve than the easy one which is a purely engineering problem. People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea.
> People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea
I'm not sure what you mean by this.
I think there is a pretty large consensus that our neocortex is a prediction machine (predicting future observations/outcomes from past experience), and the reason WHY it would have evolved to be this is because there is obvious massive survival benefit in successfully predicting how predators and prey will react ahead of time, what will be the outcome of your own actions, etc, etc. Prediction unlocks you from being stuck in the present having to react to things as they happen and lets you plan ahead.
Thinking = Reasoning/Planning is just multi-step prediction.
I don't think consciousness is the big deal most people think it is - it seems to be just the ability to self-observe (which helps to self-predict), but if we somehow built AGI that wasn't conscious, then who cares?
Not why it was created, why the systems in the brain lead to consciousness. Your option B requires understanding not just mechanically how but the fundamental reason for why consciousness appears. If we just understand the mechanics all we can confidently do is work toward a more and more accurate representation of the brain. AGI without consciousness is speculated but hard for me to believe in.
As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input.
I would expect that all intelligent animals are conscious, and any AI we build with a roughly brain-like architecture in terms of connections, looping, and being prediction based would also report itself to be conscious and describe a similar subjective experience. LLMs seem much too simple (just layer-wise pass-thru data flow) to be conscious.
It's possible that some of the neural connections supporting consciousness may have evolved, or been enhanced, due to the evolutionary value of enhanced self-prediction (i.e. this is the reason), but as noted I expect it basically "comes for free" with any complete enough cognitive/sensory architecture.
> As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input.
As far as I know, consciousness is referring to something other than self-referential systems, especially with regards to the hard problem of consciousness.
The [philosophical zombie](https://en.wikipedia.org/wiki/Philosophical_zombie) thought experiment is well-known for imagining something with all the structural characteristics that you mention but without conscious experience as in "what-its-like" to be someone.
It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.
When considering the structural underpinnings of consciousness, it's interesting to note the phenomena of "blindsight", which is essentially a loss of visual consciousness without an actual loss of vision!
Note that anything with mental access to it's own deliberations and sensory inputs will by definition always be able to report "what it's like" to be themselves - what they are experiencing (what's in their mind). If something reports to you their quales of vision or hearing, isn't this exactly what we mean by "what it's like" to be them - how they feel they are experiencing the world?!
> It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.
Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience. We can certainly have an opinion or belief or assume that sufficient structure means consciousness, which is a perfectly reasonable stance to take and one that many would take, but we have to be careful to understand that's not a scientific stance since it isn't testable or falsifiable, which is why it's been called the "hard problem" of consciousness. It's an unfounded belief we choose out of reasons like psychological comfort.
With regards to your latter point, I think you are making some sophisticated distinctions regarding the "map and territory" relation, and it seems you've hit upon the crux of the matter: how can we report "what its like" for us to experience something the other person hasn't experienced, if its not deconstructible to phenomenal states they've already experienced (and therefore constructible for them based off of our report)? The landmark paper here is "What Is It Like to Be a Bat?" by Josh Nagel, and if you're ever curious it's a pretty short read.
With regards to "blindsight" since I'm not familiar with it and curious, how do we distinguish between loss of visual consciousness and loss of information transfer between conscious regions, or loss of memory about conscious experience?
I'm not sure how much, if any, work has been done to study the brains of people with blindsight. I'm also not sure I would differentiate between loss of visual consciousness and loss of information transfer ... my understanding is that it's the loss if information transfer that is causing the loss of consciousness (e.g maybe your visual cortex works fine, so you can see, and you can perform some visual tasks that have been well practiced and/or no longer need general association cortex, but if the connection between visual cortex and association cortex was lost, then perhaps this is where you become unaware of your ability to see, i.e. lose visual consciousness).
I don't think it's a memory issue - one classic test of blindsight is asking the patient to navigate a cluttered corridor full of obstacles, which the patient succeeds in doing despite reporting themselves as blind - so it's a real-time phenomena, not one of memory.
> Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience.
That seems to come down to defining, in a non hand-wavy way, what we mean by "conscious phenomenal experience". If this is referring to personal subjective experience, then why is just asking them to report that subjective experience unsatisfactory ?!
I get that consciousness is considered as some ineffable personal experience, but as a thought experiment, what if the experimenter, defining themselves as "conscious" wanted to probe if some subject's subjective experience differed from their own, then they could at least attempt to verbalize any and all aspects of their own (the experimeter's) subjective experience and ask the subject if they felt the same, and the more (unconstrained) questions they asked without finding any significant difference would make it asymptotically unlikely that there was any difference.
> which is why it's [p-zombie detection] been called the "hard problem" of consciousness
AFAIK the normal definition of the hard problem is basically how and why the brain gives rise to qualia and subjective experience, which really seems like a non-problem...
We have thoughts and emotions, and mental access to these, so it has to feel like something to be alive and experience things. If we introspect on what having, say, vision, is like, or what it is like to have our eyes open vs shut, then (assuming we don't have blindsight!) we are obviously going to experience the difference and be able to report it - it does "feel" like something.
Qualia are an interesting thing to discuss - why do we experience what we do, or experience anything at all for that matter when we see, say a large red circle. Why does red feel "red"? Why and how does music feel different in nature to color, and why does it feel the way it does, etc?
I think these are also really non-problems that disappear as soon as you start to examine them! What are the differences in quales of seeing a small red circle vs a large red circle, or a large blue circle vs a large red one... When you consider differences in quales vs the fact that we experience anything at all (which is proved by our ability to report that we do). Color is perceived as surface attribute with a spatial extent, with colors differentiated by what they remind us of. Blue brings to mind water, sky and other blue things, Red brings to mind fire, roses, and other red things. Perception of color can be proven to be purely associative, not absolute, by Ivo Kohler's chromatic adaptation experiments, having the subject wear colored goggles whose effect "wears off" after a few days with normal subjective perception of color returning.
I'm actually curious here, because maybe our experiences are different. When you look at something red, before any associations or thoughts kick in, before you start thinking "this reminds me of fire" or analyzing it, is there something it's like for that redness to be there? Some quality to it that exists independent of what you can say about it?
For me, I can turn off all the thinking and associations and just... look. And there's something there that the looking is of or like, if that makes sense. It's hard to put into words because it's prior to words, and can possibly be independent of them.
But maybe that's not something universal? I know some people don't have visual imagery or an inner voice, so maybe phenomenal experience varies more than we assume. Does that distinction between the experience itself and your ability to think/talk about it track for you at all?
> And there's something there that the looking is of or like, if that makes sense
I think I know what you mean, but if you consider something really simple like the patch of a single color, even without any color associations (although presumably they are always there subconsciously) then isn't the experience just of "a surface attribute, of given spatial extent". There is something there, that is the same in that spatial region, but different elsewhere.
At least, that's how it seems to me, and isn't that exactly how the quale of a color has to be - that is the essence of it ?!
> then isn't the experience just of "a surface attribute, of given spatial extent".
I don't know why this seems to be so hard for me to think about and even put into words, but isn't "the experience of the surface attribute of a given spatial extent" something other than the experience of the surface attribute of a given spatial extent itself?
I mean that the words we use to describe something aren't the something itself. Conceivably, you can experience something without ever having words, and having words about a phenomenal visual experience doesn't seem to change the experience much or at all (at least for me).
Maybe another way of phrasing this would be something like: can we talk about red blotches using red blotches themselves, in the same way that we can talk about words using words themselves? And then, supposing that we could talk about red blotches using red blotches (maybe the blotches are in the form of words or structured like knowledge, I dunno), can we talk about red blotches without ever having experienced red blotches? I learned this idea from Mary's Room thought experiment, but I still don't know what to think about it.
Yes - the experience / quale has nothing to do with words.
The point (opinion) I'm trying to make is that something like the quale of vision, that is so hard to describe, basically has to be the way it is, because that is it's fundamental nature.
Consider starting with your eyes closed, and maybe just a white surface in front of you, then you open your eyes. Seeing is not the same as not-seeing, so it has to feel different. If it was a different color then the input to your brain would be different, so that has to feel different too. Vision is a spatial sense - we have a 2-D array of rods and cones in our retina feeding into our brain, so (combined with persistence of vision) we experience the scene in front of us all at once in a spatial manner, completely unlike hearing which is a temporal sense with one thing happening after another... etc, etc.
It seems to me that when you start analyzing it, everything about the quale of vision (or hearing, or touch, or smell) has to be the way it is - it is no mystery - and an artificial brain with similar senses would experience it exactly the same way.
Yep that's a cogent, serious stance, and it sounds a lot like illusionism (famously argued by Daniel Dennett) or functionalism if you ever wanted to check out more about it.
It’s a serious stance, but the really interesting thing to me here is that its not a settled fact. What’s quite surprising and unique about this field is that unlike physics or chemistry where we generally agree on the basics, in consciousness studies you have some quite brilliant minds totally deadlocked on the fundamentals. There is absolutely no consensus on whether the problem is 'solved' or 'impossible,' and its definitely not a matter of people not taking this seriously enough or making some rash judgments or simple errors.
I find this fascinating because this type of situation is pretty rare or unique in modern science. Maybe the fun part is that I can take one stance and you another and here there's no "right answer" that some expert knows and one of us is "clearly" wrong. Nice chatting with you :)
Correct - the vast majority of people vastly underestimate the complexity of the human brain and the emergent properties that develop from this inherent complexity.
exactly, the brain - what a concept! over here you have broca's area, there, wernicke, then Bowman's crest, sector 19, and undiscovered country.
if you put the brain in the shape of a tube you'd have a really long err, well, let's say it's not a good idea to do that. the brain gives me goosepimples, my brain too
I think it's more an algorithm problem. I've been reading how LLMs work and the brain does nothing like matrix multiplication over billions of entities. It seems a very inefficient way to do it in terms of compute use, although efficient in terms of not many lines of code. I think the example of the brain shows one could do far better.
Quantum compute is my guess. Being able to switch entire models at atomic speeds will give the perception of intelligence at least. There is still a lot there that will need to be figured out between now and then.
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
Dial-up modems can transfer a 4K HDR video file, or any other arbitrary data.
It obviously wouldn't have the bandwidth to do so in a way that would make a real-time stream feasible, but it doesn't involve any leap of logic to conclude that a higher bandwidth link means being able to transfer more data within a given period of time, which would eventually enable use cases that weren't feasible before.
In contrast, you could throw an essentially unlimited amount of hardware at LLMs, and that still wouldn't mean that they would be able to achieve AGI, because there's no clear mechanism for how they would do so.
From modern perspective it's obvious that simply upping the bandwidth allows streaming high-quality videos, but it's not strictly about "more bigger cable". Huge leaps in various technologies were needed for you to watch video in 4k:
- 4k consumer-grade cameras
- SSDs
- video codecs
- hardware-accelerated video encoding
- large-scale internet infrastructure
- OLED displays
What I'm trying to say is that I clearly remember reading an old article about sharing mp3s on P2P networks and the person writing the article was confident that video sharing, let alone video streaming, let alone high-quality video streaming, wouldn't happen in foreseeable future because there were just too many problems with that.
If you went back in time just 10 years and told people about ChatGPT they simply wouldn't believe you. They imagined that an AI that can do things that current LLMs can do must be insanely complex, but once technology made that step, we realized "it's actually not that complicated". Sure, AGI won't surface from simply adding more GPUs into LLMs, just like LLMs didn't emerge from adding more GPUs to "cat vs dog" AI. But if technology took us from "AI can tell apart dog and cat 80% of the time" to "AI is literally wiping out entire industry sectors like translation or creative work while turning people into dopamine addicts en masse" within ten years, then I assume that I'll see AGI within my lifetime.
There's nothing about 4K videos that needs an SSD, an OLED display, or any particular video codec, and "large-scale internet infrastructure" is just a different way of saying "lots of high-bandwidth links". Hardware graphics acceleration was also around long before any form of 4K video, and a video decoding accerator is such an obvious solution that dedicated accelerators were used for early full-motion video before CPUs could reasonably decode them.
Your anecdote regarding P2P file sharing is ridiculous, and you've almost certainly misunderstood what the author was saying (or the author themselves was an idiot). That there wasn't sufficient bandwidth or computing power to stream 4K video at consumer price points during the heyday of mp3 file sharing, didn't mean that no one knew how to do it. It would be as ridiculous as me today saying that 16K stereoscopic streaming video can't happen. Just because it's infeasible today, doesn't mean that it's impossible.
Regarding ChatGPT, setting aside the fact that the transformer model that ChatGPT is built on was under active research 10 years ago, sure, breakthroughs happen. That doesn't mean that you can linearly extrapolate future breakthroughs. That would be like claiming that if we developer faster and more powerful rockets, then we will eventually be able to travel faster than light.
There are some tradeoffs in the other direction. Digital neurons can have advantages that biological neurons do not.
For example, if biology had a "choice" I am fairly confident that it would have elected to not have leaky charge carriers or relatively high latency between elements. Roughly 20% of our brain exists simply to slow down and compensate for the other 80%.
I don't know that eliminating these caveats is sufficient to overcome all the downsides, but I also don't think we've tried very hard to build experiments that directly target this kind of thinking. Most of our digital neurons today are of an extremely reductive variety. At a minimum, I think we need recurrence over a time domain. The current paradigm (GPU-bound) is highly allergic to a causal flow of events over time (i.e., branching control flows).
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
Me too. But, I worry this “want” may not be realistic/scalable.
Yesterday, I was trying to get some Bluetooth/BLE working on a Raspberry CM 4. I had dabbled with this 9 months ago. And things were making progress then just fine. Suddenly with a new trixie build and who knows what else has changed, I just could not get my little client to open the HCI socket. In about 10 minutes prompt dueling between GPT and Claude, I was able to learn all about rfkill and get to the bottom of things. I’ve worked with Linux for 20+ years, and somehow had missed learning about rfkill in the mix.
I was happy and saddened. I would not have k own where to turn. SO doesn’t get near the traffic it used to and is so bifurcated and policed I don’t even try anymore. I never know whether to look for a mailing list, a forum, a discord, a channel, the newsgroups have all long died away. There is no solidly written chapter in a canonically accepted manual written by tech writers on all things Bluetooth for the Linux Kernel packaged with raspbian. And to pile on, my attention span driven by a constant diet of engagement, makes it harder to have the patience.
It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
While I generally agree with this, I have mixed feelings. On the one hand, AI could be smart enough to reach the "enlightened master engineer" and can reach super-human levels of...simplification. In some ways, complexity can result from improper layering and abstraction inversion. It takes a holistic view to realize that the lines (i.e. interfaces) between layers were drawn improperly and redesign everything together, achieving an overall simplification.
A good example is the web platform. It's just enormous...to the point that no human can really understand how it all even works. And I say that as someone who worked for a long time on a narrow part of that stack (V8). It being only a little over a million lines of code, it is incredibly intricate and subtle, because it implements a pretty weird language, has lots of optimizations, advanced GC, multiple compilers, etc. And that's just the JS engine. Add in the layout engine, rendering engine, multi-process architecture...it's beyond the comprehension a single mind.
We're not yet at the level that an AI can understand code really deeply yet, but may we will reach the point where an AI understands enough of it and can code competently enough to start over from scratch and build something we can both understand and does the things we actually want it to do.
I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
Distros do have manuals, they just usually come in the form of user-curated wikis these days. ArchWiki is usually my first stop when I run into a Linux issue, even as a fellow Debian user.
> It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
This is the real AI risk we should be worried about IMO, at least short term. Information technology has made things vastly more complicated. AI will make it even more incomprehensible. Tax code, engineering, car design, whatever.
It's already happening at my work. I work at big tech and we already have a vast array of overly complicated tools/technical debt no one wants to clean up. There's several initiatives to use AI to prompt an agent, which in turn will find the right tool to use and run the commands.
It's not inconceivable that 10 or 20 years down the road no human will bother trying to understand what's actually going on. Our brains will become weaker and the logic will become vastly more complicated.
Yes, I'm already doing it. But the problem is there's not a lot of incentive from management to do it.
Long term investment in something that can't easily be quantified is a non-starter to management. People will say "thank you for doing that" but those who create new features that drive metrics get promoted.
That's why it's very important to work for an organization run by engineers or former engineers when writing software.
And engineers who actually shipped and maintained successful products over time.
For software developed in-house by non-software organizations, the incentives are all wrong because management cannot properly assess the value of well architected systems.
I think LLMs as a replacement for Google, Stack Overflow, etc. is a no brainer. As long as you can get to the source documents when you need them, and train yourself to sniff out hallucinations.
(We already do this constantly in categorizing human generated bullshit information and useful information constantly. So learning to do something similar with LLM output is not necessarily worse, just different.)
What's silly at this point is replacing a human entirely with an LLM. LLMs are still fundamentally unsuited for those tasks, although they may be in the future with some significant break throughs.
Yeah, using LLMs makes me reconsider the complexity of the software I'm producing and I'm relying on. In a sense LLMs can be a test for the complexity and the fast iteration cycles could yield better solutions than the existing ones
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
I figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
"Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
"It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
Because the depth of concept is so deep, and if you're just beginning then it's going to take a while for that statement to illuminate how obvious it is.
For those who've been sniffing this since early 2010, it's so blindly obvious they've already dropped llms on the floor and moved onto deeper alternative research.
For the rest of us, we're still catching coke bottles from the sky and building places of worship around them
> Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end.
There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
> I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it.
Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes.
So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
Well it kind of sucks honestly because im never going to get any sort of recognition, but that's partly by choice and partly because I don't have the right personal connections. I 100% understand why all of you think I'm a significant fool. Thats ok. Like i said, im burying the idea. But its obvious enough that someone else will discover it soon enough.
You literally see no irony here? You are doubling, tripling down on some idea you won't even vaguely describe to people, but you are so utterly convinced that it's right you don't appear to even admit the possibility that your idle thoughts could be wrong, or just silly. Frankly it sounds delusional and you should question yourself more often. If you're so afraid of being right that you won't even write the idea down to have another person look at it for fear it will somehow set the world on fire because of its rightness, then you should seriously consider talking to a therapist. If that comes across as an ad hominem, it isn't, and it isn't meant as an insult or a dismissal of your idea--you've actually offered no idea. Frankly, you should be a little embarrassed as you've done nothing but grouse about how smart you are, and people react negatively to that. Puffing gets us nowhere except drawing attention to yourself and gave us no new ideas or thoughts to discuss, except your ego. I'd recommend you question yourself more and talk to some people you trust about whatever it is you think you discovered.
I was hired on to help run these experiments with a machine they were trying to perfect. They wanted to do all this stuff with running two machines in parallel, dumping the refuse from one into another machine, all kinds of silly stuff.
I looked at it and in a few minutes I sketched in my notebook a solution to their problem that would achieve the maximum theoretical yield without all these wacky experiments.
But then what would be my job? I was hired to run these experiments not solve their whole problem in 5 minutes. So I didn't tell them. I was getting paid so i didn't care if they were wasting time. I'm a smiley little trans woman anyway, they wouldn't have listened. I did work out the math though on my own.
A few months later, a team of phds at a university we were working with hit on a rudimentary form of the same idea, so at that point I revealed to the boss, oh, look I've worked out the math for that already. And I wrote up the code, and we hit the maximum theoretical yield and it ran 100x faster computationally, and i got a pat on the back. But the reason they listened was because a man with a degree said it.
Why would I bother giving them ideas if they won't listen anyway? If i had given them the answer months earlier, I would not have gotten any kind of bonus even in the unlikely event that I had been taken seriously.
So now of course youre all telling me im insane, and thats fine because honestly I have no incentive to give you answers ahead of time. Go suck eggs. Figure shit out on your own.
> If i had given them the answer months earlier, I would not have gotten any kind of bonus even in the unlikely event that I had been taken seriously.
You guaranteed that you wouldn't get recognition by keeping the idea to yourself. Your victimization was by your own design, and it stayed where it started: in your head. And ironically you were the one being passive agressive and showing bias towards your coworkers.
Ugh why do you care so much? Leave me alone. The big insane statement i made was "i have some idea of what the next bit of advancement will look like". Transformers are a special case of a much more beautiful general principle ok? Go away
If you're so confident that your discovery will be found by someone else soon why not just share it right here and get the credit you seem to want? Otherwise why even bring it up in the first place?
You're being downvoted for displaying the kind of overconfidence that people consider shameful.
Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
Author here. Indeed - it would be just as fantastical to deny there has been no value from deep learning, transformers, etc.
Yesterday I heard Cory Doctorow talk about a bunch of pro bono lawyers using LLMs to mine paperwork and help exonerate innocent people. Also a big win.
There's good stuff - engineering - that can be done with the underlying tech without the hyperscaling.
Not only whispr, so much of the computer vision area is not as in vogue. I suspect because the truly monumental solutions unlocked are not that accessible to the average person; i.e. industrial manufacturing and robotics at scale.
That's because industrial manufacturing and robotics are failing to bring down costs and make people's lives more affordable.
That's really the only value those technologies provide, so if people aren't seeing costs come down there really is zero value coming from those technologies.
I think a lot of AI wins are going to end up local and free much like whisper.
Maybe it could be a little bit more accurate, it would be nice if it ran a little faster, but ultimately it's 95% complete software that can be free forever.
My guess is very many AI tasks are going to end up this way. In 5-10 years we're all going to be walking around with laptops with 100k cores and 1TB of RAM and an LLM that we talk to and it does stuff for us more or less exactly like Star Trek.
Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
I will go meta into what you posted here: That people are classifying themselves as "AI skeptics". Many people are treating this in terms of tribal conflict and identity politics. On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits. If we do talk about it as a debate, we can do it when with open minds, and intellectual honesty.
I think much of this may be a reaction to the hype promoted by tech CEOs and media outlets. People are seeing through their lies and exaggerations, and taking positions like "AI/LLMs have no values or uses", then using every argument they hear as a reason why it is bad in a broad sense. For example: Energy and water concerns. That's my best guess about the concern you're braced against.
> I will go meta into what you posted here: That people are classifying themselves as "AI skeptics"
The comment you're replying to is calling other people AI skeptics.
Your advice has some fine parts to it (and simonw's comment is innocuous in its use of the term), but if we're really going meta, you seem to be engaging in the tribal conflict you're decrying by lecturing an imaginary person rather than the actual context of what you're responding to.
To me, "Tip for AI skeptics" reads as shorthand for "Tip for those of you who classify as AI skeptics".
That is why the meta commentary about identity politics made complete sense to me. It's simply observing that this discussion (like so many others) tends to go this way, and suggests a better alternative - without a straw man.
I read it more as a claim that people who advocate against AI are picking arguments as a means to an end rather than because they actually believe or care about what they're saying.
Expecting a purely technical discussion is unrealistic because many people have significant vested interests. This includes not only those with financial stakes in AI stocks but also a large number of professionals in roles that could be transformed or replaced by this technology. For these groups, the discussion is inherently political, not just technical.
I don't really mind if people advocate for their value judgements, but the total disregard for good faith arguments and facts is really out of control. The number of people who care at all about finding the best position through debate and are willing to adjust their position is really shockingly small across almost every issue.
Totally agree. It seems like a symptom of a larger issue: people are becoming increasingly selfish and entrenched in their own bubbles. It’s hard to see a path back to sanity from here.
This depends on the particular group of rationalists. An unfortunately outsized and vocal group with strong overlap in the tech community has gone to notions of quasi mathematical reasoning distorting things like EV ("expected value"). Many have stretched "reason" way past the breaking point to articles of faith but with a far more pernicious affect than traditional points of religious dogma that are at least more easily identifiable as "faith" due to their religious trappings.
Edit: See Roko's Basilisk as an example. wherein something like variation on Christian hell is independently reinvented for those not donating enough to bring about the coming superhuman AGI, who will therefore punish you- or the closest simulation it can spin up in VR if you're long gone- for all eternity. The infinite negative EV far outweighing any positive EV of doing more than subsist in poverty. Even managed to work in that it could be a reluctant, but otherwise benevolent super AI such that, while benevolent, it wanted to exist, and to maximize its chances it bound itself to a promise in the future to do these things as an incentive for people to get it to exist.
yeah maybe around the time of Archimedes it was closer to the top, but societies in which people are willing to die for abstract ideas tend to be one... where the value of life isn't quite as high as it is nowadays (ie no matter how much my inner nerd has a love and fascination for that time period, no way i'm pressing the button on any one-way time machines...).
I mean, Archimedes stands out because he searched for the truth and documented it. I'm sure most people on the planet at that time would have burned you for being a witch, or whatever fabled creature was in vogue at the time.
Only among the people who are yelling, perhaps? I find the majority of people I talk with have open minds and acknowledge the opinions of others without accepting them as fact.
> a large number of professionals in roles that could be transformed or replaced by this technology.
Right, "It is difficult get a man to understand something when his salary depends on his not understanding it."
I see this sort of irrationality around AI at my workplace, with the owners constantly droning on about "we must use AI everywhere." They are completely and irrationally paranoid that the business will fail or get outpaced by a competitor if we are not "using AI." Keep in mind this is a small 300 employee, non-tech company with no real local competitors.
Asking for clarification or what they mean by "use AI" they have no answers, just "other companies are going to use AI, and we need to use AI or we will fall behind."
There's no strategy or technical merit here, no pre-defined use case people have in mind. Purely driven by hype. We do in fact use AI. I do, the office workers use it daily, but the reality is it has had no outward/visible effect on profitability, so it doesn't show up on the P&L at the end of the quarter except as an expense, and so the hype and mandate continues. The only thing that matters is appearing to "use AI" until the magic box makes the line go up.
I've heard the same breathless parroting of the marketing hype at large O(thousands ppl) cloud tech companies. A quote from leadership:
> This is existential. If we aren't early adopters of AI tools we will be left behind and will never catch up.
This company is dominant in the space they operate in. The magnitude of the delusion is profound. Ironically, this crap is actually distracting and affects quality, so it could affect competitiveness--just not how they hope.
I've seen the same trend. AI neeeds to be everywhere, preferably yesterday, but apart from hooking everything up to an LLM withot regards for the consequences nobody seems to know what the AI is supposed to do.
Politics is the set of activities that are associated with making decisions in groups, or other forms of power relations among individuals, such as the distribution of status or resources.
Most municipalities literally do not have enough spare power to service this 1.4 trillion dollar capital rollout as planned on paper. Even if they did, the concurrent inflation of energy costs is about as political as a topic can get.
Economic uncertainty (firings, wage depression) brought on by the promises of AI is about as political as it gets. There's no 'pure world' of 'engineering only' concerns when the primary goals of many of these billionaires is leverage this hype, real and imagined, into reshaping the global economy in their preferred form.
The only people that get to be 'apolitical' are those that have already benefitted the most from the status quo. It's a privilege.
Hear hear, It's funny having seen the same issue pop up in video game forums/communities. People complaining about politics in their video games after decades of completely straight faced US military propaganda from games like Call of Duty but because they agree with it it wasn't politics. To so many people politics begins where they start to disagree.
There are politics and there are Politics, and I don't think the two of you are using the same definition. 'Making decisions in groups' does not require 'oversimplifying issues for the sake of tribal cohesion or loyalty'. It is a distressingly common occurrence that complex problems are oversimplified because political effectiveness requires appealing to a broader audience.
We'd all be better off if more people withheld judgement while actually engaging with the nuances of a political topic instead of pushing for their team. The capacity to do that may be a privilege but it's a privilege worth earning and celebrating.
My definition is the definition. You cannot nuance wash the material conditions that are increasing tribal polarization. Rising inequality and uncertainty create fear and discontent, people that offer easy targets for that resentment will have more sway.
The rise of populist polemic as the most effective means for driving public behavior is also downstream from 'neutral technical solutions' designed to 'maximize engagement (anger) to maximize profit'. This is not actually a morally neutral choice and we're all dealing with the consequence. Adding AI is fuel for the fire.
I would rather not trust the first person who claims <outgroup> wants to starve me. Polemnics may be legitimate - they may not be, I haven't thought about it deeply - but they are undoubtedly worth dropping from my own information diet.
I mean, it is intellectually honest to point out that the AI debate at the point is much more a religious or political than strictly technical really. Especially the way tech CEOs hype this as the end of everything.
> IMO the move is drop the politics, and discuss things on their technical merits.
I'd love this but it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
It's not unlike when religious people condemn a book they refuse to read. The merits of the book don't matter, it's symbolic opposition to something broader.
Okay, but a lot of people are calling environmental and content theft arguments "political" in an attempt to make it sound frivolous.
It's fine if you think every non-technical criticism against AI is overblown. I use LLMs, but it's perfectly fine to start from a place of whether it's ethical, or even a net good, to use these in the first place.
People saying "ignoring all of those arguments, let's just look at the tech" are, generously, either naive or shilling. Why would we only revisit these very important topics, which are the heart of how the tech would alter our society, after it's been fully embraced?
Well they're separate issues. Someone could plausibly take the position that air travel should be banned for environmental reasons, but that has no relevance to the utility of air travel. If a group of people were loudly proclaiming that planes were not only bad but useless, anyone who routinely uses planes would obviously find them non-credible.
They're not separate at all, especially if the question is "How hard should we push people to use this."
To be clear, there are a lot of people who routinely used airplanes who don't post-covid, but insisted that they had to. Yeah, I think it's pretty wasteful to fly across the country for a 30 minute meeting. Most don't fly at all. I don't know what mass-psychosis white collar industries were under to think that was necessary.
They're 100% separate. "Planes aren't useful" and "planes don't work" are completely different sentiments than "it's pretty wasteful to fly across the country for a 30 minute meeting" and "we shouldn't push people hard to use air travel".
I know for a fact that planes work, because I've been on a plane and observed it lifting me high off the ground and rapidly transporting me to a distant location. The fact that planes typically emit CO2 doesn't make their existence and utility some kind of mass hallucination.
This distinction may sound a bit silly, because I assume we all agree that planes literally work. But the point I'm making is as it applies to AI. Like many people, I know from experience that AI isn't vaporware and is extremely useful for many purposes. I'm sure many others haven't had the same experience for various reasons, and factually report their observations in good faith — but that's different from pushing a narrative which one wishes to be true, regardless of how valid the reasons for that wish may be.
> I know for a fact that planes work, because I've been on a plane and observed it lifting me high off the ground and rapidly transporting me to a distant location. The fact that planes typically emit CO2 doesn't make their existence and utility some kind of mass hallucination.
You're arguing with an imaginary person. Read what I wrote. I didn't call AI vaporware, I said we shouldn't consider its integration into society purely on technical merits, ignoring the cost, which I think could be big if OpenAI's very public plans are made reality. You're making a strawman.
The mass hallucination is not that plane's are useful, it's that a plane is the only reasonable solution to human communication.
Honestly, you're just further illustrating the complete erosion of nuance that comes when you paint people with concerns about AI as frivolous.
Driving a massive truck in the city is stupid too and most short flights should be replaced with high speed rail. And AI wastes a monumental amount of resources.
> The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
I keep seeing arguments like this. They sound like a bit like a form of nihilsm. Do you really think we shouldn't worry about risks to the environment simply because we're all hypocrites on that front in one way or another? I get the frustration and have been guilty of using this type of argument myself in the past, but refusing to discuss a problem because the people raising the concern are imperfect human beings doesn't seem like a tenable position.
Charitably, I think you can read into that a not-unreasonable (if unproven) assertion that there are many lower hanging fruits on the tree that would do endlessly more good for the cause to pick than data centers, and AI at least has the arguable potential upside of alleviating some of those specific burdens-- better health care and less environmental pollution through various improved forms of automation. Or at least when addressing the stub claim of the sort in the GP comment, you should assume these fairly straightforward subclaims pre-emptively and respond to this stronger form of the argument. It saves time, at least if you're going to seek out a discussion it does. Plenty of counter claims to them, but it gets the conversational ball rolling in a productive direction and if the response in turn is less constructive then you also know not to bother any more.
I think that form of argument is called "whataboutism". Whether flights waste energy or are environmentally unfriendly is really a separate issue. Both things can be bad.
I wouldn't ignore those arguments but most of the time, they're so poorly formed (eg. using data without logic), they aren't really worth listening to. If you believe AI provides no value, then any environmental cost is too high for you but you can't convey that by trying to dramatize how high it is. That's dishonest and I think people rightly turn off it.
Right now Open"AI", Oracle and everyone else are burning billions of dollars to buy and run these llms, they raise the price of energy around them, they provide negative economic benefit. It's dishonest of you to pretend that isn't the case.
We’ve all used the tools, and they’re… fine. They probably will contribute modestly to overall productivity in certain fields, but they certainly aren’t as transformative or magical as the current hype suggests. I’m not sure why you insist that we continue to fawn over these things.
No, we've used it, you are creating a strawman argument assuming "AI skeptics" are illiterate and/or incapable of understanding. You ironically are the one refusing to accept the possibility that you are wrong.
> No, we've used it, you are creating a strawman argument
There exists a class of "ai-skeptic" who proudly proclaim they have never and will never use AI. Examples are not hard to find, though I see them more on reddit/instagram/bluesky than I do on HN.
If that does not describe you then my comment is not about you.
Maybe you've used it-- but a very large number of the AI skeptic comments I see that actually cite particular experiences, even comments in the pages of HN, amount to things like, "ChatGPT hallucinated when I asked about the local price of product X and if it was in stock anywhere around. How can anyone take LLM and AI seriously?"
Or worse, things like "Real science and real engineering doesn't rely on tools that behave randomly.".
> it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
Why? Would you say the same if the topic was about recreational drugs? Or, to bring it closer to home, if the topic was about social media?
I think you're being disingenuous by making the analogy to religious people refusing to read a certain book. A book is a foundational source of information. OTOH, one can be informed about GenAI without having used GenAI; you can study the math behind the model, the transformer architecture, etc---the foundational sources of information on this topic. If our goal is to "drop the politics, and discuss things on their technical merits" well I don't see how it can get more purely technical than that.
The frustrating thing is when you're debating people who firmly believe that generative AI "has no utility"... but also refuse to ever try it themselves.
(Which they might even justify because they've read the transformer paper or whatever. That doesn't help inform you if these things actually have practical applications!)
Yep. Same for the other direction: there is a very strong correlation between identity politics and praising AI on Twitter.
Then there's us who are mildly disappointed on the agents and how they don't live their promise, and the tech CEOs destroying the economy and our savings. Still using the agents for things that work better, but being burned out for spending days of our time fixing the issues the they created to our code.
The adoption and use of technology frequently (even typically) has a political axis, it's kind of just this weird world of consumer tech/personal computers that's nominally "apolitical" because it's instead aligned to the axis of taste/self-identity so it'll generate more economic activity.
> On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits.
Zero obligation to satisfy HN audience; tiny proportion of the populace. But for giggles...
Technical merits: there are none. Look at Karpathy's GPT on Github. Just some boring old statistics. These technologies are built on top of mathematical principles in textbooks printed 70-80 years ago.
The sharding and distribution of work across numerous machines is also a well trodden technical field.
There is no net new discovery.
This is 100% a political ploy on the part of tech CEOs who take advantage of the innumerate/non-technical political class that holds power. That class is bought into the idea that massive leverage over resource markets is a win for them, and they won't be alive to pay the price of the environmental destruction.
It's not "energy and water" concerns, it's survival of the species concerns obfuscated by socio-political obligations to keep calm carry on and debate endlessly, as vain circumlocution is the hallmark of the elders whose education was modeled on people being VHS cassettes of spoken tradition, industrial and political roles.
IMO there is little technical merit to most software. Maps, communication. That's all that's really needed. ZIRP era insanity juiced the field and created a bunch of self-aggrandizing coder bros whose technical achievements are copy-paste old ideas into new syntax and semantics, to obfuscate their origins, to get funded, sell books, book speaking engagements. There is no removing any of this from politics as political machinations gave rise to the dumbest era of human engineering effort ever.
The only AI that has merit is robotics. Taking manual labor of people that are otherwise exploited by bougie first worlders in their office jobs. People who have, again with the help of politicians, externalized their biologies real needs on the bodies of poorer illiterates they don't have to see as the first-world successfully subjugated them and moved operations out of our own backyard.
Source: was in the room 30 years ago, providing feedback to leadership how to wind down local manufacturing and move it all over to China. Powerful political forces did not like the idea of Americans having the skills and knowledge to build computers. It ran afoul of their goals to subjugate and manipulate through financial engineering.
Americans have been intentionally screwed out of learning hands on skills with which they would have political leverage over the status quo.
There is no removing politics from this. The situation we are in now was 100% crafted by politics.
Hey Simon, author here (and reader of your blog!).
I used to share your view, but what changed my mind was reading Hao's book. I don't have it to hand, but if my memory serves, she writes about a community in Chile opposing Google building a data centre in their city. The city already suffers from drought, and the data centre, acccording to Google's own assessment, would abstract ~169 litres of water a second from local supplies - about the same as the entire city's consumption.
If I also remember correctly, Hao also reported on another town where salt water was being added to municipal drinking water because the drought, exacerbated by local data centres, was so severe.
It is indeed hard to imagine these quantities of water but for me, anything on the order of a town or city's consumption is a lot. Coupled with droughts, it's a problem, in my view.
The fact that certain specific data centres are being proposed or built in areas with water issues may be bad, but it does not imply that all AI data centres are water guzzling drain holes that are killing Earth, which is the point you were (semi-implicitly) making in the article.
Just because it doesn’t leave the cycle doesn’t mean it’s not an issue. Where it comes back down matters and as climate change makes wet places wetter and dry places drier, that means it’s less distributed
That said, the water issue is overblown. Most of the water calculation comes from power generation (which uses a ton) and is non-potable water.
The potable water consumed is not zero, but it’s like 15% or something
The big issue is power and the fact that most of it comes from fossil fuels
>The big issue is power and the fact that most of it comes from fossil fuels
This has almost zero to do with the data centers themselves and instead the politicians we vote in.
Simply put we need more 'clean' power generation any way you go about it. Economic growth and production is based on ones ability to produce power. We've been coasting on increasing efficiency for a long time, but we've been in need of larger sources of power and distribution for decades, data centers or not.
With that said DC's shouldn't be built in west texas where it's dry. East of the Mississippi gets enough rain that you build a reservoir and you'll have more than enough water to feed the DC for decades.
Is it more of a pricing problem? If data centers paid the same for water as I do they’d be far more efficient with it. Likewise for power. Giving sweet deals to these things based on promises of jobs and tax revenue seems a bad idea at this point.
The way they measure water consumption is genuinely unbelievably misleading at best. For example measuring the water evaporated from a dams basin if any hydroelectric power is used.
Counting water is genuinely just asinine double counting ridiculousness that makes down stream things look completely insane. Like making a pound of beef look like it consumes 10,000L of water.
In reality of course running your shower for 10 to 15 hours is no where near somehow equivalent to eating beef lasagna for dinner and we would actually have a crisis if people started applying any optimization pressure on these useless metrics.
My favourite example is the complaints about “e-waste” because AirPod batteries are not replaceable. Never mind that the batteries make up about half of the weight of the things ro begin with, the entire annual production of AirPods and their cases would barely fill a single shipping container.
No, not a container ship, or anything substantial like that, just one container (1).
Yet, it makes people lose their rational minds and start foaming at the mouth about Apple’s “wasteful practices”.
Apologies, looks like I mis-remembered the numbers from the original debate, the actual number is about 110x 40-foot containers. It can be calculated easily enough from the # shipped, the volume of a charging case, and the volume of a shipping container.
It doesn’t alter the conclusion that much: in the grand scheme of human industrial activity, this is nothing.
I.e.: if AirPods were made by an independent company, that firm would be in the top 50 of the largest corporations in the world!
For example, a comparable sized company would be Coca Cola, which goes through 300,000 tons of aluminium annually for their cans, not to mention oil used for plastic bottles!
Drinking water does not magically appears in the water cycle the next day.
[0] - "And what we found is is that up to 43% of data centers, and this is our largest data centers, are located in areas of high or extremely high water stress. And that's really shocking because data centers require huge amount of drinking water to be able to cool their servers."
"I took all your money and gambled it away, but don't worry because it wasn't destroyed, it's still circulating in the economy and will come back someday."
A reduction in what can be used from the summer snowmelt is a problem, regardless of whether equivalent atoms are redeposited in the winter snowfall.
I’m sure there was some planning commission process involved with the development of these sites. I’m curious if anyone has bothered to look at those meeting minutes to see if there are some material misrepresentation of the water and power needs. I’m going to guess that answer is no.
The mistake you are making is letting the author choose your points of comparison, without having a high-level picture of where water usage goes. Comparing water usage to a city is misleading because cities don't use much water; large-scale water use is entirely dominated by agriculture.
I'm conflicted. Zooming out, the problem isn't with AI specifically but economic development in general. Everything has a side effect.
For decades we've been told we shouldn't develop urban centers because of how it development affects local communities, but really it just benefited another class of elites (anonymous foreign investors), and now housing prices are impoverishing younger generations and driving homelessness.
Obviously that's not a perfect comparison to AI, which isn't as necessary, but I think the anti-growth argument isn't a good one. Democracies need to keep growing or authoritarian states will take over who don't care so much about human rights. (Or, authoritarian governments will take over democracies.)
There needs to be a political movement that's both pro-growth and pro-humanity, that is capable of making hard or disruptive decisions that actually benefits the poor. Maybe that's a fantasy, but again, I think we should find ways to grow sustainably.
Not just the poor, how about the bottom 99%? This is what's so frustrating to me about the culture wars and identity politics. Regardless of ones views on the hot button cultural issue du jour, at best they are a distraction, and at worst actively exploited by moneyed interests as a political smokescreen to prevent changes that would be obvious wins for super majorities of the population if analyzed and viewed through a more sober and objective lens of the net effects.
IIRC Google chose to pull out altogether to punish the locals for standing up to them— even though they happily built air-cooled data centers elsewhere.
I mean yes, almost all corporations that have a choice do this. Walmart is one of the better known ones that will put a store right on the edge of a municipality that doesn't want one and cause all kinds of issues for the city at hand.
Nestle is and has been 10000x worse for global water security than all other companies and countries combined because nobody in the value chain cares about someone else’s aquifer.
It’s a social-economic problem of externalities being ignored , which transcends any narrow technological use case.
What you describe has been true for all exported manufacturing forever.
I think the point is: where does this end? Do we continue to build orders-of-magnitude bigger models guzzling orders-of-magnitude more water and other resources, in pursuit of the elusive AGI?
At some we need to end this AGI" rat race and focus on realizing practical benefits from the models we currently have.
My interpretation was "If an industry that actively works to harm the global health of humanity through their addictive and unhealthy food products is using way more water and we're OK with it, maybe we should give a pass to the industry using a fraction of that water to improve human productivity."
Ton of nuance in my characterizations of both industries, of course, but to a first approximation they are accurate.
Well, the food industry continues churning billions in profits at the expense of our health, so statistically speaking, looks like "we" are OK with!
Totally agreed that criticism should be directed where it's due. But what this thread is saying is that criticism of GenAI is misdirected. I haven't seen nearly as much consternation over e.g. the food industry as I'm seeing over AI -- an industry that increasingly looks like its utility exceeds its costs.
(If the last part sounds hypothetical, in past comments I've linked a number of reports, including government-affiliated sources, finding noticeable benefits from GenAI adoption.)
I should have been more precise with my terms, but there is a difference between "food" and the "food industry" indicated by the likes of Nestle. Yes, everybody needs food. No, nobody needs the ultraprocessed junk Nestle produces.
I didn't see the OP's point as whataboutism, but rather putting things into perspective. We are debating the water usage of a powerful new technology that a large fraction of the world is finding useful [1], which is a fraction of what other, much more frivolous (golf courses!) or even actively harmful (Nestle!) industries use.
[1] https://news.ycombinator.com/item?id=45794907 -- Recent thread with very rough numbers which could well be wrong, but the productivity impact is becoming detectable at a national level.
Attempt generosity. Can you think of another way to interpret the comment above yours? Is it more likely they are calling their own argument a red herring, or the one they are responding to?
If something looks like a "weird stance", consider trying harder to understand it. It's better for everyone else in the conversation.
Just because there are worse abuses elsewhere doesn't mean datacenters should get a pass.
Golf and datacenters should have to pay for their externalities. And if that means both are uneconomical in arid parts of the country then that's better than bankrupting the public and the environment.
> I asked the farmer if he had noticed any environmental effects from living next to the data centers. The impact on the water supply, he told me, was negligible. "Honestly, we probably use more water than they do," he said. (Training a state-of-the-art A.I. requires less water than is used on a square mile of farmland in a year.) Power is a different story: the farmer said that the local utility was set to hike rates for the third time in three years, with the most recent proposed hike being in the double digits.
The water issue really is a distraction which harms the credibility of people who lean on it. There are plenty of credible reasons to criticize data enters, use those instead!
The other reason water usage is a bad thing to focus on is that datacenters don't inherently have to use water. It's not like servers have a spigot where you pour water in and it gets consumed.
Water is used in modern datacenters for evaporative cooling, and the reason it's used is to save energy -- it's typically around 10% more energy efficient overall than normal air conditioning. These datacenters often have a PUE of under 1.1, meaning they're over 90% efficient at using power for compute, and evaporative cooling is one of the reasons they're able to achieve such high efficiency.
If governments wanted to, they could mandate that datacenters use air conditioning instead of evaporative cooling, and water usage would drop to near zero (just enough for the restrooms, watering the plants, etc). But nobody would ever seriously suggest doing this because it would be using more of a valuable resource (electricity / CO2 emissions) to save a small amount of a cheap and relatively plentiful resource (water).
"...So to wrap up, I misread the Berkeley Report and significantly underestimated US data center water consumption. If you simply take the Berkeley estimates directly, you get around 628 million gallons of water consumption per day for data centers, much higher than the 66-67 million gallons per day I originally stated..."
163.7 billion / 43,070 billion * 100 = 0.38 - less than half a percentage point.
It's very easy to present water numbers in a way that looks bad until you start comparing them thoughtfully.
I think comparing data center water usage to domestic water usage by people living in towns is actually quite misleading. UPDATE: I may be wrong about this, see following comment: https://news.ycombinator.com/item?id=45926469#45927945
The vast majority of water in agriculture goes to satisfy our taste buds, not nourish our bodies. Feed crops like alalfa consume huge amounts of water in the desert southwest but the desert climate makes it a great place to grow and people have an insatiable demand for cattle products.
We could feed the world with far less water consumption if we opted not to eat meat. Instead, we let people make purchasing decisions for themselves. I'm not sure why we should take a different approach when making decisions about compute.
> We could feed the world with far less water consumption if we opted not to eat meat.
If you look at the data for animals, that’s not really true. See [1] especially page 22 but the short of it is that the vast majority of water used for animals is “green water” used for animal feed - that’s rainwater that isn’t captured but goes into the soil. Most of the plants used for animal feed don’t use irrigation agriculture so we’d be saving very little on water consumption if we cut out all animal products [2]. Our water consumption would even get a lot worse because we’d have to replace that protein with tons of irrigated farmland and we’d lose the productivity of essentially all the pastureland that is too marginal to grow anything on (50% of US farmland, 66% globally).
Animal husbandry has been such a successful strategy on a planetary scale because it’s an efficient use of marginal resources no matter how wealthy or industrialized you are. Replacing all those calories with plants that people want to actually eat is going to take more resources, not less, especially when you’re talking about turning pastureland into productive agricultural land.
That paper makes the opposite argument than you thought it made. Even from a freshwater perspective it’s more water-efficient to get calories/protein/fat directly from crops than from animal products.
Since you like their work, the authors of your paper answered that question more generally here https://link.springer.com/article/10.1007/s10021-011-9517-8 where they conclude "The water footprint of any animal product is larger than the water footprint of crop products with equivalent nutritional value".
You make some often debunked claims, like we'd have to plant more crops to feed humans directly if we stopped eating meat.
This shouldn't make intuitive sense to you since animals eat feed grown on good cropland (98% of the water footprint of animal ag) that we could eat directly, and we lose 95% of the calories when we route crops through animals.
You’re confusing my argument. I’m not arguing that meat is more efficient than plants; that by itself is obviously untrue because of the trophic efficiency loss. I’m arguing that meat uses marginal resources that would otherwise be useless for growing food. If we eliminate meat we’d eliminate a lot of marginally productive farmland that is naturally irrigated and have to replace it with industrialized irrigation farming that will be significantly more expensive and use more water and energy. Maybe if we had a global cultural reset with industrialized farming, but no one is going to be happy replacing their beef and chicken with the corn and barley and hay that those lands normally grow, much of it inedible to humans.
That paper isn’t actually debunking anything that I’m saying. If the water foot print per calorie is 20x for beef but the feed is grown with 90% of its water from rainfall, that’s not a 20x bigger footprint in a way that practically matters because most of that water is unrecoverable anyway. The water that is recoverable just makes it through the watershed.
Meat is a way to convert land that cant grow things people can or want to eat into things that people will eat. That pastureland and marginal cropland growing animal feed can’t just be converted to grow more economically productive crops like fruit and vegetables without Herculean engineering effort and tons of water and fertilizer. Instead the farming would have to stress other fertile ecosystems like the Southwest which would make the water problems worse, even if their total “footprint” is smaller. The headline that beef uses 20x more water per calorie completely ignores where that water comes from and how useful it actually is to us.
I don’t doubt that we can switch to an all plant diet as a species but people vastly underestimate the ecological and societal cost to do so.
I mean it's even simpler. Almonds are entirely non essential (many other more water efficient nuts) to the food supply and in California consume more water than the entire industrial sector, and a bit more than all residential usage (~5 million acre-feet of water).
Add a datacenter tax of 3x to water sold to datacenters and use it to improve water infrastructure all around. Water is absolutely a non-issue medium term, and is only a short term issue because we've forgotten how to modestly grow infrastructure in response to rapid changes in demand.
Growing almonds is just as essential as building an AI. Eating beef at the rate americans do is not essential. Thats where basically all the water usage is going.
Iran's ongoing water crisis is an example. One cause of it is unnecessary water-intensive crops that they could have imported or done without (just consume substitutes).
It's a common reasoning error to bundle up many heterogeneous things into a single label ("agriculture!") and then assign value to the label itself.
I am surprised by your analytical mistake of comparing irrigation water with data-center water usage...
They are not equivalent. Data centers primarily consume potable water, whereas irrigation uses non-potable or agricultural-grade water. Mixing the two leads to misleading conclusions on the impact.
That's a really good point - you're right, comparing data center usage to potable water usage by towns is a different and more valid comparison than comparing with water for irrigation.
They made a good point, but keep in mind that they're doing a "rules for thee, not for me" sometimes.
The same person who mentioned potable water being an important distinction also cited a report on data center water consumption that did not make the distinction (where the 628M number came from).
The factual soundness of my argument is independent of the report quality :-) the report influences comprehension, not correctness...
The fact data centers are already having a major impact on the public water supply systems is known, by the decisions some local governments are forced to do, if you care to investigate...
"...in some regions where data centers are concentrated—and especially in regions already facing shortages—the strain on local water systems can be significant. Bloomberg News reports that about two-thirds of U.S. data centers built since 2022 are in high water-stress areas.
In Newton County, Georgia, some proposed data centers have reportedly requested more water per day than the entire county uses daily. Officials there now face tough choices: reject new projects, require alternative water-efficient cooling systems, invest in costly infrastructure upgrades, or risk imposing water rationing on residents...."
What counts as data center water consumption here? There are many ways to arguably come up with a number.
Does it count water use for cooling only, or does it include use for the infrastructure that keeps it running (power generation, maintenance, staff use, etc.)
Is this water evaporated? Or moved from A to B and raised a few degrees.
This is the real point. Just measuring the amount of water involved makes no sense. Taking 100 liters of water from a river to cool a plant and dumping them back in a river a few degrees warmer is different from taking 100 liters from a fossil acquifer to evaporatively cool the same plant.
A farmer is a valuable perspective but imagine asking a lumberjack about the ecological effects of deforestation, he might know more about it than an average Joe, but there's probably better people to ask for expertise?
> Honestly, we probably use more water than they do
This kind of proves my point, regardless of the actual truth in this regard, it's a terrible argument to make: availability of water starts to become a huge problem in a growing amount of places, and this statement implies the water usage of something, that in basic principle doesn't need water at all, uses comparable amount of water as farming, which strictly relies on water.
I think the point here is that objecting to AI data center water use and not to say, alfalfa farming in Arizona, reads as reactive rather than principled. But more importantly, there are vast, imminent social harms from AI that get crowded out by water use discourse. IMO, the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology.
On the flip side, the crypto hype machine pretty seamlessly flipped to the AI hype machine, so it makes sense the same anti crowd shifted pretty seamlessly. Given the practical applications of crypto were minimal and the externalities were mostly crime and pollution, I’m not at all surprised that many people expect the same for AI.
The anti-crypto people were correct, though. Why should we not push back when we’re seeing the same type of baseless hype that surrounded crypto being cultivated around the AI space?
They were and we should push back and yes, there is a mountain of baseless hype. But if you train your fire on the wrong thing, you risk not addressing the actual problem.
But if I say "I object to AI because <list of harms> and its water use", why would you assume that I don't also object to alfalfa farming in Arizona?
Similarly, if I say "I object to the genocide in Gaza", would you assume that I don't also object to the Uyghur genocide?
This is nothing but whataboutism.
People are allowed to talk about the bad things AI does without adding a 3-page disclaimer explaining that they understand all the other bad things happening in the world at the same time.
Because your argument is more persuasive to more people if you don't expand your criticism to encompass things that are already normalized. Focus on the unique harms IMO.
If you take a strong argument and through in an extra weak point, that just makes the whole argument less persuasive (even if that's not rational, it's how people think).
You wouldn't say the "Uyghur genocide is bad because of ... also the disposable plastic crap that those slave factories produce is terrible for the environment."
Plastic waste is bad but it's on such a different level from genocide that it's a terrible argument to make.
Adding a weak argument is a red flag for BS detectors. It's what prosecutors do to hoodwink a jury into stacking charges over a singular underlying crime.
I don't think there's a world where a water use tax is levied such that 1) it's enough for datacenters to notice and 2) doesn't immediately bankrupt all golf courses and beef production, because the water use of datacenters is just so much smaller.
We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Beef, I guess, is a popular type of food. I’m under the impression that most of us would be better off eating less meat, maybe we could tax water until beef became a special occasion meal.
I'm saying that if you taxed water enough for datacenters to notice, beef would probably become uneconomical to produce at all. Maybe a good idea! But the reason datacenters would keep operating and beef production wouldn't is that datacenters produce way more utility per gallon.
You can easily write a law that looks like this:
There is now a water usage tax. It applies only to water used for data-centers. It does not apply to residential use, agricultural use, or any other industrial use.
We do preferential pricing and taxing all the time. My home's power rate through the state owned utility is very different than if I consumed the exact same amount of power, but was an industrial site. I just checked and my water rate at home is also different than if I were running a datacenter. So in all actuality we already discriminate for power and water based on end use. at least where I live. Most places I have lived have different commercial and residential rates.
In other words, the price of beef can stay the same.
Yes, if you don't like datacenters, tax datacenters. If you want to do it via a tax on the water they use then I guess that works.
But the environment doesn't really care whether the water is being used by a datacenter or something else. My point is just that data centers are actually more efficient users of water compared to many less-controversial users.
>In other words, the price of beef can stay the same.
And yet, if you believe the environmentalist argument in the first place, the price of beef should go up for the damages it causes. Hence why a lot of people think the people complaining about AI are wearing an environmentalist mask, rather than having an actual care about the environment.
Pigouvian taxes are fine, but should be applied across all sources of damage.
A lot of beef is produced in such a way that taxing municipal water won't make a material difference. Even buying market rate water rights in the high desert, which already happens in beef production, is a pretty small tariff on the beef.
> We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Might as well get rid of all the lawns and football fields while we’re at it.
Water taxes should probably be regional. The price of water in the arid Southwest is much higher than toward the East coast. You might see both datacenters and beef production moving toward states like Tennessee or Kentucky.
Of course golf courses are useful as shown by the fact that people pay to use them. Perhaps you mean that you personally haven't (yet) found them useful, but you know that different people want different things. I think eating shrimp is disgusting and never eat them but I don't want to ban global shrimp production because the people it would harm are not me!
My perspective from someone who wants to understand this new AI landscape in good faith. The water issue isn't the show stopper it's presented as. It's an externality like you discuss.
And in comparison to other water usage, data centers don't match the doomsday narrative presented. I know when I see it now, I mentally discount or stop reading.
Electricity though seems to be real, at least for the area I'm in. I spent some time with ChatGPT last weekend working to model an apples:apples comparison and my area has seen a +48% increase in electric prices from 2023-2025. I modeled a typical 1,000kWh/month usage to see what that looked like in dollar terms and it's an extra $30-40/month.
Is it data centers? Partly yes, straight from the utility co's mouth: "sharply higher demand projections—driven largely by anticipated data center growth"
With FAANG money, that's immaterial. But for those who aren't, that's just one more thing that costs more today than it did yesterday.
Coming full circle, for me being concerned with AI's actual impact on the world, engaging with the facts and understanding them within competing narratives is helpful.
It's not even an externality? They just pay market price for water. You can argue the market price is priced badly (e.g., maybe prices are set by the state), but that doesn't make it an externality. The benefits/costs are still accrued by (and internal to) buyer and seller.
If datacenters are getting electricity and water at a rate lower than retail (costs passed on to residents or tax payers), and factors like noise and water pollution aren't factored in, then yes there are unpriced externalities.
> If any region has a lot of freshwater and little potable water, the best way to make potable water more available and cheaper is to introduce a new large buyer, which will give the local utility enough revenue to upgrade and expand their treatment facilities. Saying that my data is misleading because Al "only uses valuable potable water" actually gets the issue backwards: adding demand for more potable water in regions with lots of freshwater makes potable water cheaper and more abundant for everyone else per unit.
1. That just because a region doesn’t have enough potable water to support humans and data centers, it also doesn’t have enough potable water to support the humans alone.
2. That the temporary increase in water prices due to the new demand of the data centers will provide enough revenue to upgrade its facilities
3. Even given enough revenue to upgrade its facilities, that the utility will choose to upgrade its facilities and increase demand
4. That the downsides of a temporary increase in water prices while new facilities are built is acceptable and will not cause suffering
5. Even after new facilities are built, that the cost of those facilities will be low enough and the increase in supply large enough that water prices for humans will be lower than they were originally, even with a large and wealthy new buyer on the market.
It doesn’t feel like a very strong argument to me.
Just as an example, Loudoun County has the most data center concentration in the world by a wide margin, and also has significantly lower water prices than the Virginia average. Seems like they've been able to build out huge new water capacity for data centers and that hasn't raised househould prices, or even kept them low. https://andymasley.substack.com/i/171855599/the-county-with-...
This part of his argument is not self evident or intuitive and I'm not convinced that the abstract economic model maps cleanly to the messy reality. I'm much more assuaged by the fact that it seems to cost ~$1/1000 gallons to convert fresh water into potable water.
Like, if agriculture uses fresh water and data centers use potable water, the important question is how hard it is to convert fresh water into potable water?
The answer seems to be "not very" so the difference is kind of moot
There's a bunch in there but this bit caught my eye in particular:
> The US public water supply uses ~40 billion gallons per day, all of this is potable. Data centers used 50 million gallons per day onsite in 2023. So their potable water usage was 0.13% of the public water supply.
That's a naive argument. Infrastructure construction timelines are typically measured in years and decades. You need to find the political will to do it and sufficient guarantees of long-term demand to justify the investment. And the work itself is often done in difficult environments, such as under major streets or on privately owned land that may already be in use.
In parts that do not care much about property rights and citizens are not allowed to challenge government decisions in the courts.
If I was trying to build new infrastructure in Finland, I would add four years to the project between the planning and the construction stages. Two years for urban planning and public hearings, and another two years for the inevitable lawsuits. Other Western countries should have similar delays.
Farmland, AI data centers, and golf courses do not provide the same utility for water used. You are not making an argument against the water usage problem, you are only dismissing it.
Almonds are pretty cherry picked here as notorious for their high water use. Notably, we're not betting an entire economy and pouring increasing resources into almond production, either. Your example would be even more extreme if you chose crops like the corn used to feed cattle. Feeding cows alone requires 21.2 trillion gallons per year in the US.
The people advocating for sustainable usage of natural resources have already been comparing the utility of different types of agriculture for years.
Comparatively, tofu is efficient to produce in terms of land use, greenhouse gas emissions, and water use, and can be made shelf-stable.
>Almonds are pretty cherry picked here as notorious for their high water use.
If water use was such a dire issue that we needed to start cutting down on high uses of it, then we should absolutely cherry pick the high usages of it and start there. (Or we should just apply a pigouvian tax across all water use, which will naturally affect the biggest consumers of it.)
Yes, that's roughly what I said in my post. If we're doing a controlled economy and triaging for the health of the ecosystem, we'd start with feed for cattle, and almonds wouldn't be much further down on the list.
The contention with AI water use is that something like this is currently happening as local water supplies are being diverted for data-centers.
People have been sounding the alarm about excessive water diverted to almond farming for many years though, so that doesn't really help the counter-argument.
Aren't Californian almonds like 80% of the world's production?
Is the US AI data-centers producing 80% of the world's IT ?
I ask legitimately, I think that would already make it more apples to apples.
Also if you ask me personally, I'd rather have almonds than cloud AI compute. Imagine a future 100 years from now, we killed the almonds, never to be enjoyed ever again by future generations... Or people don't have cloud AI compute. It's personal, but I'd be more sad that I'd never get to experience the taste of an almond and all the cuisine that comes with it.
> Is the US AI data-centers producing 80% of the world's IT
You've misread it. It's not compared to AI datacenters, it's every type of datacenter, for all types of computing.
In the future scenario you've laid out it wouldn't be cloud AI compute. You wouldn't be able to use HN or send email or pay with a credit card or play video games or stream video.
AI has way more utility than you are claiming and less utility than Sam Altman and the market would like us to believe. It’s okay to have a nuanced take.
I'm more unhappy than happy, as there are plenty of points about the very real bad side of AI that are hurt by such delusional and/or disingenuous arguments.
That is, the topic is not one where I have already picked a side that I'd like to win by any means necessary. It's one where I think there are legitimate tradeoffs, and I want the strongest arguments on both sides to be heard so we get the best possible policies in the end.
If AI is so useful, we should have a ton of data showing an increase in productivity across many fields. GDP should be skyrocketing. We haven’t seen any of this, and every time a study is conducted, it’s determined that AI is modestly useful at best.
Well, I don't like marzipan, so both are useless? Or maybe different people find uses/utility from different things, what is trash for one person can be life saving for another, or just "better than not having it" (like you and Marzipan it seems).
ok in that case you don't need to pick on water in particular, if it has no utility at all then literally any resource use is too much, so why bother insisting that water in particular is a problem? It's pretty energy intensive, eg.
AI has no utility _for you_ because you live in this bubble where you are so rabidly against it you will never allow yourself to acknowledge it has non-zero utility.
What does it mean to “use” water? In agriculture and in data centers my understanding is that water will go back to the sky and then rain down again. It’s not gone, so at most we’re losing the energy cost to process that water.
The problem is that you take the water from the ground, and you let it evaporate, and then it returns to... Well to various places, including the ground, but the deeper you take the water from (drinking water can't be taken from the surface, and for technological reasons drinking water is used too) the more time it takes to replenish the aquifer - up to thousands of years!
Of course surface water availability can also be a serious problem.
No it’s largely the same situation I think. I was drawing a distinction between agricultural use and maybe some more heavy industrial uses while the water is polluted or otherwise rendered permanently unfit for other uses.
Other people might have other preferences. Maybe we could have a price system where people can express their preferences by paying for things with money, providing more money to the product which is in greater demand?
Sure.. Except some people / companies have so much more money, they can demand impractical things and pay above-market rates for them, causing all others to scramble to live day-to-day with the distorted market.
Right, I think a data center produces a heck of a lot more economic and human value in a year - for a lot more people - than the same amount of water used for farming or golf.
The water intensity of American food production would be substantially less if we gave up on frivolous things like beef, which requires water vastly out of proportion to its utility. If the water numbers for datacenters seem scary then the water use numbers for the average American's beef consumption is apocalyptic.
Whether people would switch off meat on their own is a separate issue. If water became scarce enough to start moving the price, then you'd absolutely see people eat less meat.
But their point does disarm the suggestion that water consumption for AI is bad because it's just for fun while meat feeds people.
Because when you eat meat, you could have eaten something far less resource intensive like tempeh. But you ate meat for reasons beyond survival. For most of us, it's because we like the taste and we're used to it.
I don't see that as having any stronger of a claim to water consumption than the things we use AI for (fun, getting work done, writing nix/k8s config) much less a claim to many times the amount of water consumption than AI data centers.
While I agree, the "meat is not sustainable" argument is literal, and evidenced in beef prices rising as beef consumption lowers over the past years. Beef is moving along the spectrum from having had been a "staple" to increasingly being a luxury.
The US never gave up eating lobster either, but many here have never had lobster and almost nobody has lobster even once a week. It's a luxury which used to be a staple.
That depends how sentient a chicken is: their brains are of similar complexity to the larger of these models, counting params as synapses.
Also, while I'm vegetarian going on vegan, welfare arguments are obviously not relevant in response to an assertation that Americans aren't going to give up meat, because if animal welfare was relevant then Americans would give up meat.
> I appreciate that you feel this way, it’ll never happen. The US will never give up on eating meat. Full stop.
I don't see any signs that the US is going to give up on AI and data centers, either. (The coming AI winter notwithstanding)
For what it's worth, I've cut back quite a bit on my beef and pork consumption, and now mostly eat chicken. The environmental and ethical arguments finally got to me.
> Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds.
Both alfalfa and almonds contain a lot of nutrients you dont find in large enough amounts (or at all) in corn and potatoes though. And alfalfa improves the soil but fixating nitrogen. Sure almonds require large amounts of water. Maybe alfalfa does as well? And of course it depends on if they are grown for human consumption or animal.
Water usage largely depends on the context, if the water source is sustainable, and if it is freshwater.
Of course water used up will eventually evaporate, and produce rainfall in the water cycle, but unfortunately at many places "fossil" water is used up, or more water used in an area then the watershed can sustainably support.
This is a constant source of miscommunication about water usage, and that of agriculture also. It is very different to talk about the water needs to raise a cow in eg. Colorado and in Scotish highlands, but this is usually removed from the picture.
The same context should be considered for datacenters.
Data center cooling towers have to use fresh rather than salt water, but they don't care about bacterial contamination or toxic traces of arsenic, antimony, and fluorine. Agriculture also has to use fresh rather than salt water. I can't think of any circumstances where water that was usable for agriculture wouldn't also be usable for cooling data centers—except when the farmer owns the water and the data center operator doesn't.
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
Not to mention that this is likely a very short-term debate. Modern AI/ML/compute has been a huge boost to fusion R&D, and tons of AI/tech capital is flowing into companies like Helion and CFS. Necessity is the mother of invention and all.
It may be a valid criticism today, but no one will be complaining about AI's environmental impact after those first few plants go live and mass production begins within the next decade. Knock on wood.
I'm skeptical about that. Helion in particular might work (https://www.youtube.com/watch?v=HlNfP3iywvI) but all the other fusion-energy companies' plans are to convert fusion energy to thermal energy and then convert the thermal energy to electrical energy in the usual way, using steam engines. That's never going to be cheaper than thermal energy powering steam engines, because thermal energy powering steam engines is part of it.
But solar energy is.
Helion's strategy is to directly convert the hot plasma's expansion into electrical energy by, basically, pushing against a powerful magnetic field. This is potentially higher efficiency than steam engines (because it's working at a higher temperature) and potentially higher power density and reliability. So it could at least theoretically get to be cheaper than steam engines. But I think it's more likely to be far more expensive for the next several decades.
I'm definitely more optimistic about Helion. It's the one that would really dramatically change the economics of power generation at scale, and they seem to have substantially engineered around major challenges like not requiring ignition and minimizing losses to Bremsstrahlung radiation. I like that they're laser-focused on launching and delivering commercial value, even if that means a bit less transparency and less focus on research demonstrations than many would prefer.
If Helion delivers Orion on anywhere close to the target timeline, it seems like they'd be primed to kick off mass production and start shipping units all over the country with minimal regulatory hurdles. With sufficiently high productive capacity, it would effectively make all other forms of power generation obsolete at utility scale. Solar would still great for small/decentralized use cases, but there wouldn't be as much reason for a power company to build new solar plants if they had the option to fill the same real estate with a bunch of Helion machines.
That being said, I'm not not optimistic about steam-based fusion power. It would be less revolutionary, but from what I understand would still be a huge advancement. My read is that it would effectively obsolete fission power and plausibly fossil fuel power as well, if not also grid-scale renewables. It would be like having SMRs with higher efficiency and no major proliferation or meltdown concerns that amplify costs. High neutron output and steam conversion add costs and complexity, but if CFS panned out and Helion didn't, I'd hardly turn my nose at that.
I could also see a world where multiple forms of fusion power pan out, and it turns out to be economical for Helion to sell tritium to other vendors like CFS rather than storing it for 12 years until it decays to He3. Maybe we'd even have colocated plants in a certain ratio where Helion provides on-site tritium generation in the exact volume needed for a D-T reactor.
https://www.helionenergy.com/articles/starting-to-build-the-... (undated) says Orion is intended to deliver 50 megawatts. If they could deliver that for under US$100 million they'd be competitive with fossil-fuel plants, assuming no fuel cost or other opex; if they could deliver it for under US$30 million they'd be competitive with solar, unless solar gets even cheaper. (It's fallen in cost by 45% over the last two years, according to https://www.solarserver.de/photovoltaik-preis-pv-modul-preis..., but almost all of that was more than a year ago; it's only fallen 10% in the last year.)
Helion's latest funding round was US$425 million, bringing total funding to over a billion, on a US$5 billion valuation. To give that money back to their investors, assuming a 40% profit margin, they'd need to sell something like 100 copies of Orion at US$30 million each, totaling 5 gigawatts. For the investors to consider them successful, it would need to be more like 2000 copies of Orion, totaling 100 gigawatts.
That's a completely plausible market size (although it's not just "all over the country", because the country they're in only has 1161 GW of generation capacity and is barely building any), so their NRE costs to date are not an impossible burden. It really depends on four things:
1. Can they hit such a low price target as US$30 million per 50-megawatt power plant? Can they ship anything for US$30 million a pop?
2. Will they be allowed to sell to China, where the customers are?
3. Will solar energy get cheaper still, forcing them to an even lower price point?
4. Will opex of the resulting reactors be manageable?
I'm not an expert on the subject, but is there an obvious bottleneck that would prevent Helion from manufacturing an arbitrarily large number of generators that scales with demand? 2000 generators doesn't seem like an overwhelmingly large number if they get paid enough to build out capacity for it, and they've already started on an assembly line manufacturing plant.
I'd expect that they probably wouldn't focus on copying Orion exactly, since Orion is just the initial PoC and will only use a single 50 MW generator. Their Nucor plant is planned to have 10x higher capacity, which seems likely to be more in line with typical deployments outside of remote areas. Opex also seems like a non-issue to me, given that the machines will be automated, the fuel will be cheap, and they won't have the same safety-critical need for strict regulation and human oversight as fission plants.
My understanding is that exporting might not be a huge hurdle to get a green light for due to the low proliferation concerns, although I wouldn't be surprised if Helion itself had some reservations around that or blocked it on designing some sort of tamper-resistant enclosure for IP protection purposes.
But I also don't see a reason why the US couldn't be a highly lucrative market for Helion, particularly given the increasing demand from AI. Solar in China today benefits from major alignment of interests and political tailwinds. Similarly, fusion could cut through partisan divides and NIMBYism that have hamstrung power buildout in the US. The left will buy in for climate benefits, the right will buy in for energy independence, both will buy in for economics + practicality, and NIMBYs will buy in for lower electric bills with minimal risks or drawbacks. Whether or not directly publicly funded, there will at least be every incentive to collectively get out of private industry's way and allow mass fusion deployment with minimal regulatory burdens.
That said, I'm sure solar will continue to get cheaper, and iron-air batteries from companies like Form Energy seem to be an extremely promising solution for high-scale storage. I also like the idea of using Prometheus Fuels to repurpose existing fossil fuel plants as storage for renewables. But there are still theoretical limits on the efficiency of solar panels, and space usage and intermittency could make it less cost-efficient than fusion in many if not most cases. We can't really make an apples-to-apples comparison between a mature technology in active high-volume production and a speculative future technology, but the latter in this case does seem to me to have a higher ceiling on its potential value over the long run.
> I'm not an expert on the subject, but is there an obvious bottleneck that would prevent Helion from manufacturing an arbitrarily large number of generators that scales with demand?
No, nothing is obvious to anyone outside the company.
> I'd expect that they probably wouldn't focus on copying Orion exactly,
Hopefully their next model will come out before another billion dollars of investment goes down the drain.
> Opex also seems like a non-issue to me, given that the machines will be automated, the fuel will be cheap,
Yes, that could happen, but what it really means is that we don't have any idea what the opex is.
> My understanding is that exporting might not be a huge hurdle to get a green light for due to the low proliferation concerns,
I think the bigger concern there would be whether China would import it.
> But I also don't see a reason why the US couldn't be a highly lucrative market for Helion,
Well, the US hasn't been a highly lucrative market for power plants for 50 years. Maybe the advent of new, radically cheaper technology could change that, but I suspect it won't; the advent of new, radically cheaper energy in the late 18th century didn't turn China or Japan into a highly lucrative market for power plants, for essentially social or political rather than technical and economic reasons.
> We can't really make an apples-to-apples comparison between a mature technology in active high-volume production and a speculative future technology, but the latter in this case does seem to me to have a higher ceiling on its potential value over the long run.
Yes, I agree. But we weren't talking about the long run, but rather about currently existing companies that need to return money to their investors in the next ten or twenty years. Breakthroughs do happen, but rarely.
Yeah, there are lots of unknowns and valid reasons to be pessimistic or optimistic. I lean toward cautious optimism based on the information that is available, but of course no one knows the future until it happens.
Just to clarify, Orion is one site, not a particular generator model. Orion working would be the green light to build and ship more generators, not a sign that they'd need to go back to the drawing board on an "Orion v2".
re: opex, we do have some idea of what it will be. We know that deuterium is cheap, we know that fusion produces low waste, we know that extensive safety protocols to protect against meltdown are not required, we know that preventing theft of fissile materials is not a security concern, and we know that these machines are substantially automated. We may not have precise balance sheets, but it's also not entirely opaque or mysterious.
I'm not sure the examples of historical China and Japan apply. They didn't invent coal power and then decide not to use it (although Japan was nevertheless fairly quick to industrialize). Fusion as of now seems likely to reach commercialization first in the US, and there's currently a lot of demand for power here with insufficient supply to meet projected growth. It's a very different market from when we were actively shipping our high-power-consumption industries overseas.
I don't agree that breakthroughs happen "rarely". I mean, we wouldn't be here in this thread without the massive breakthrough of useful LLMs, which were considered sci-fi before ChatGPT's release less than three years ago; fusion had the breakthrough of ignition at NIF just a few days after that; and we're only in the position of considering solar and batteries as a competitive energy source due to many small breakthroughs that have occurred in recent history. Breakthroughs happen all the time, but we quickly forget their significance and let them become boring.
One of the biggest problems in environmental education (imo) is the lack of notion regarding the footprint of products and services we consume daily; from the water and CO2 costs of a meal, of a heated pool, of a car, etc. It is eye-opening.
I first came across this type of info with the book "How Bad Are Bananas", from Mike Berners-Lee. I really enjoyed it, and I just saw that the new edition even includes stuff like hosting a World Cup, data centers, and space tourism!
It should give a good foundation to start talking about it.
Water location matters. Is the data center in a desert with scarce potable water for locals? Or is next to a large Canadian lake, plenty of potable water, with people who want to trade something for currency so they can put avocados in their salad?
A lot of data centers are near the Columbia river, as power is cheap there thanks to hydroelectric; which flows through an arid desert-like region, but is also the largest river in the western US and it's simply impossible to pump too much water out of it.
Another issue is that you could, in principle, build data centers in places where you don't need to evaporate water to cool them. For example, you could use a closed loop water cooling system and then sink that heat into the air or into a nearby body of water. OVH's datacenter outside Montreal¹ does this, for example. You can also use low carbon energy sources to power the data center (nuclear or hydro are probably the best because their production is reliable and predictable).
Unlike most datacenters, AI datacenters being far away from the user is okay since it takes on the order of seconds to minutes for the code to run and generate a response. So, a few hundred milliseconds of latency is much more tolerable. For this reason, I think that we should pick a small number of ideal locations that have a combination of weather that permits non-sub-ambient cooling and have usable low carbon resources (either hydropower is available and plentiful, or you can build or otherwise access nuclear reactors there), and then put the bulk of this new boom there.
If you pick a place with both population and a cold climate, you could even look into using the data center's waste heat for district heating to get a small new revenue stream and offset some environmental impact.
>that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
Some time ago, I read the environmental impact assessment for a proposed natural gas thermal power plant, and in it they emphasized that their water usage was very low (to the point that it fit within the unused part of the water usage allowance for an already existing natural gas thermal power plant on the same site) because they used non-evaporative cooling.
What prevents data centers from using non-evaporative cooling to keep their water usage low? The water usage argument loses a lot of its relevant in that case.
In europe several power plants get shut down each summer because the heated water from those plants would have significant impact on the local wildlife.
I went down that “water use” rabbit hole a month ago and basically… it’s just a bunch of circular reporting that was based on some paper published in 2023[1]. For ChatGPT 3.5 they claimed “500ml for 10-50 responses”. In 2024, Washington Post published an article that took their claim and said “519 milliliters per email”[2] but didn’t source that from the original paper… that “shocking headline” took off and got widely circulated and cited directly, treating the WaPo calculation as if it were the original research finding. Then tech blogs and advocacy sites ran with it even harder, citing each other instead of any actual research[3].
If you look at the original paper they are quite upfront with the difficulty of estimating water use. It’s not public data—in fact it’s usually a closely held trade secret, plus it’s got all kinds of other issues like you don’t know where the training happened, when it happened, what the actual cooling efficiency was, etc. The researchers were pretty clear about these limitations in the actual paper.
Basically, it’s urban legend at this point. When OpenAI’s CEO later said ChatGPT uses ~0.3ml per query, that’s roughly 100x less than the viral claims.
The problem is more one of scale: a million liters of water is less than half of a single Olympic-sized swimming pool. A single acre of alfalfa typically requires 4.9 - 7.6 million liters a year for irrigation. Also, it's pretty easy to recycle the data center water, since it just has to cool and be sent back, but the irrigation water is lost to transpiration and the recycling-by-weather process.
So, even if there's no recycling, a data center that is said to consume "millions" rather than tens or hundreds of millions is probably using less than 5 acres of alfalfa in consumption, and in absolute terms, this requires only a swimming-pool or two of water per years. It's trivial.
I think the source is the bigger problem. If they take the water from sources which are already scarce, the impact will be harsh. There probably wouldn't be any complaints if they would use sewerage or saltwater from the ocean.
> Also, it's pretty easy to recycle the data center water, since it just has to cool
Cooling and returning the water is not always that simple. I don't know specifically about datacentres, but I know about wasting clean water in other areas, cooling in power plants, industry, etc. and there it can have a significant impact on the cycle. At the end it's a resource which is used at least temporary, which has impact on the whole system.
> If they take the water from sources which are already scarce, the impact will be harsh.
Surprised I had to scroll down this far to see this mentioned.
The water use argument is highly local and depends on where we are building these data centers. Are you building in the great lakes region with plenty of fresh water and no water scarcity issues (yet)? Ok fine.
But we aren't building there. We're building in Arizona, Nevada, Nebraska, Iowa putting further stress in an area that water scarcity is already an issue, or soon going to become one due to long term drought conditions. Or Texas, which already has problems with their power grid.
We're building in these locations because they're cheap. If we're going do to this, we need to not let the bottom line be the sole driving decision of data center locations. If it's not profitable to build elsewhere, don't build it until you've figured out how to make it efficient enough to where it is profitable.
Not really. The majority of data center water withdrawal (total water input) is consumed ("lost" to evaporation etc...) with a minority of it discharged (returned in liquid form). I believe it's on the order of 3/4ths consumed, but that varies a lot by local climate and cooling technology.
There's lots of promising lower-consumption cooling options, but seems like we are not yet seeing that in a large fraction of data centers globally.
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing) 177-comment discussion on a small component of the author's argument. The author's argument is an important one that hardly hinges at all on water usage specifically, given the vast human and financial capital invested in LLM buildout so far.
"Presented by the USGA" (the United States Golf Association) gave me a wry chuckle there.
That said, here are the relevant numbers from that 2012 article in full:
> Most 18-hole golf facilities utilize surface waters (ponds, lakes) or on-site irrigation wells. Approximately 14 percent of golf facilities use water from a public municipal source and approximately 12 percent use recycled water as a source for irrigation.
> Specific water sources for 18-hole courses as indicated by participants are noted below:
> 52 percent use water from ponds or lakes.
> 46 percent use water from on-site wells.
> 17 percent use water from rivers, streams and
creeks.
> 14 percent use water from municipal water systems.
It depends on the region. In dryer regions most golf courses are already on greywater system. Irrigation mainly targets fairways, teeboxes, and greens, while the rough is allowed to go to hell in the dryer months.
Some golf courses also are designed to spread floodwater and protect surrounding development. Golf is sort of a happy bonus on top of that.
Private courses are usually far better irrigated than their public counterparts where even the fairway is liable to be left to dry out.
I did some napkin math on data center water usage for a 500MW data center in the Columbia River valley.
It uses as much water per year as 200 acres of alfalfa in California’s Central Valley. There are around 1M acres of alfalfa growing in California.
2.5MW of data center capacity is roughly equal to 1 acre of irrigated alfalfa in water usage. If you’re pulling fossil aquifer water, open loop evaporative cooling may not be the best idea, but there are plenty of places east of 100 degrees west in the US that have virtually ‘unlimited’ water where cooling towers are a great idea since they almost double the COP of a chilled water system.
On the water front, in my area agriculture flood irrigates with pristine aquifer water, while surface water gets treated and dumped into the drinking supply. This is due to the economics of historic water rights.
The AI water usage aspect is pretty clearly a lie and a gross misunderstanding at best
https://open.substack.com/pub/andymasley/p/the-ai-water-issu...
There are dozens of other things that use we use everyday that have a larger impact.
I think there a real concerns here but the water usage argument is a poor one
Water and power are local issues. And data center use of water and power is already, currently having local impact on politics. I saw ads about it during the election cycle that just concluded. Candidates had to answer questions about it at debates and in interviews.
People are using these arguments for the simple reason that they demonstrably resonate with average people who live near data centers.
They probably don’t resonate with people who have plenty of income and/or do not compete with data centers locally for resources.
The water argument rings a bit hollow for me not due to whataboutism but more that there's an assumption that I know what "using" water means, which I am not sure I do. I suspect many people have even less of an idea than I do so we're all kind of guessing and therefore going to guess in ways favorable to our initial position whatever that is.
Perhaps this is the point, maybe the political math is that more people than not will assume that using water means it's not available for others, or somehow destroyed, or polluted, or whatever. AFAIK they use it for cooling so it's basically thermal pollution which TBH doesn't trigger me the same way that chemical pollution would. I don't want 80c water sterilizing my local ecosystem, but I would guess that warmer, untreated water could still be used for farming and irrigation. Maybe I'm wrong, so if the water angle is a bigger deal than it seems then some education is in order.
If water is just used for cooling, and the output is hotter water, then it's not really "used" at all. Maybe it needs to be cooled to ambient and filtered before someone can use it, but it's still there.
If it was being used for evaporative cooling then the argument would be stronger. But I don't think it is - not least because most data centres don't have massive evaporative cooling towers.
Even then, whether we consider it a bad thing or not depends on the location. If the data centre was located in an area with lots of water, it's not some great loss that it's being evaporated. If it's located in a desert then it obviously is.
If you discharge water into a river, there are environmental limits to the outlet temperature (this is a good thing btw). The water can't be very hot. That means you need to pump a large volume of water through because you can only put a small amount of energy into each kg of water.
If you evaporate the water on the other hand, not only is there no temperature limit but it also absorbs the latent heat of vaporisation. The downside is it's a lot more complex and also the water is truly consumed rather than just warming it up.
Put that way, any electricity usage will have some "water usage" as power plants turn up their output (and the cooling pumps) slightly. And that's not even mentioning hydroelectric plants!
Water can range from serious concern to NBD depending on where the data center is located, where the water is coming from, and the specific details of how the data center's cooling systems are built.
To say that it's never an issue is disingenuous.
Additionally one could image a data center built in a place with a surplus of generating capacity. But in most cases, it has a big impact on the local grid or a big impact on air quality if they bring in a bunch of gas turbines.
I think the water use arguments are relevant, particularly in regions of the world and US (CA) where potable water is scarce, but land and electricity are available .
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
The reported case about water wells running dry had to do with issues in construction rather than anything about the data center's regular operation:
> But the reason their taps ran dry (which the article itself says) was entirely because of sediment buildup in groundwater from construction. It had nothing to do with the data center’s normal operations (it hadn’t begun operating yet, and doesn’t even draw from local groundwater). The residents were wronged by Meta here and deserve compensation, but this is not an example of a data center’s water demand harming a local population.
The nice thing about the data center water usage panic is that whenever someone appeals to it, I immediately know that either they haven't done their homework or they're arguing in bad faith.
We really need to stop tying people's retirement to the market. I've already lost ground due to 2008, and COVID, and while I was young, I lived through my parents suffering through dotcom as well.
It's long past time we have a universal public pension, funded at least in part with a progressive wealth tax, or least go back to private defined benefit pensions to where the employer or pension fund bears the risk rather than the individual.
Supplement it with market speculation if you want, but we need something guaranteed for everyone that's a little more robust and provides a better living than the paltry social security we have now.
skip water discussion because it's just irrelevant. If you can debunk AGI then of course we should stop spending trillions on it. If you can't debunk AGI then water usage is just a nonfactor.
Arguments in isolation are usually poor. The water usage arguments usually (always?) comes along with a bunch of other arguments, including power consumption, workers rights, consumer protection, public safety, enshittifcation, etc.
When evaluating the economical cost or morality of a thing, (just like when training a machine learning model) the more data you consider the more accurate the result (although just like statistical modelling it is worth to be wary of overfitting).
- People sceptical of transformer ANNs directly leading to AGI within any reasonable period are also sceptical of transformer ANNs directly leading to AGI any time in the far future
This kind of generalisations don't help you as the huge number of comments underneath yours likely shows
I don't think anyone who read my comment here misunderstood my usage of the term "AI skeptic" as applying to any form of machine learning as opposed to modern generative AI.
As for food production; that might be important? IDK, I am not a silicon "intelligence" so what do I know? Also, I have to "eat". Wouldn't it be a wonderful world if we can just replace ourselves, so that agriculture is unnecessary, and we can devote all that water to AGI.
TIL that the true arc of humanity is to replace itself!
Given the difference in water usage, more data centers does not mean less water for agriculture in any meaningful way.
If you genuinely want to save water you should celebrate any time an acre of farm land is converted into an acre of data center - all the more water for the other farms!
Farms already produce more than enough food to feed everyone (and, indeed, the excess is a feature because food security is really important). The reason not everyone is fed is not due to needing to divide water resources between farms and other uses.
Going only by the effective need of humans is a bad argument. A lot of farmers wouldn't survive without subsidies and are not vital to our food supply.
Stop eating beef. With the water saved we can grow enough food for any realistic human population. Ok we solved this one. Or do humans NEED burgers as well? We can already feed all people, any starvation is strictly a political problem not a food existing on the planet problem
We produce enough food for everyone already, and then waste a huge amount of it. Our food problem isn't about producing more, it's about distributing what we have.
Yes, it is worthwhile to ask how much value we get.
And a whole bunch of us are saying we don't see the value in all these datacenters being built and run at full power to do training and inference 24/7, but you just keep ignoring or dismissing that.
It is absolutely possible that generative AI provides some value. That is not the same thing as saying that it provides enough value to justify all of the resources being expended on it.
The fact that the amount of water it uses is a fraction of what is used by agriculture—which is both one of the most important uses humans can put water to, as well as, AIUI, by far the single largest use of water in the world—is not a strong argument that its water usage should be ignored.
That's a fair point. For people already finding great utility in GenAI it seems like a foregone conclusion, but for others it can be understandably frustrating that people take it as an article of faith that "AI is worth it."
Disclosure, I'm very much in the former camp, but I try to ground myself with broader empirical evidence. I've found an increasing amount of empirical evidence that GenAI is providing value more than commensurate with its costs.
Largely the studies I've looked at focus on productivity boosts, I guess because that is very easy to tie to economic impact. This recent thread has some relevant sources and extremely rough numbers, but the outcome seems to be that for a 1% increase in datacenter usage we may have gotten a 1.2% boost in national productivity: https://news.ycombinator.com/item?id=45794907
Now, this absolutely not a rigorous analysis, but could be a good starting point for how to think about these tradeoffs.
it is also a fraction of golf courses which you again ignore. this is just typical "don't do anything!!" ism. there's no argument here.. even if data centres used .00001 millilitre of water you would say the same thing.
Oh, I think golf courses shouldn't exist. They're awful in a number of ways. You want to play golf? VR or minigolf.
But (as I pointed out elsewhere in this discussion [0]) why should I have to mention everything that uses water in a way I think is detrimental in order to be allowed to talk about why I think this thing uses water in a way that is detrimental?
Why would the logic extend there? If golf courses really do use a lot of water, energy, etc, and they largely benefit a small subset of people, it doesn’t make a ton of sense for them to exist.
Of course making the world a better place for everyone is very against western and American exceptionalism and freedom.
Eventually people stop building more data centers as food becomes scarce and expensive, and farms become the hot new thing for the stock market, cereal entrepreneurs become the new celebrities and so on. Elon Husk, cereal magnate.
It's not about it being scary, its about it being a gigantic, stupid waste of water, and for what? So that lazy executives and managers can generate their shitty emails they used to have their comms person write for them, so that students can cheat on their homework, or so degens can generate a video of MLK dancing to rap? Because thats the majority of the common usage at this point and creating the demand for all these datacenters. If it was just for us devs and researchers, you wouldn't need this many.
Whether it's a "gigantic" waste of water depends on what those figures mean. It's very important to understand if 25 million liters of water per year is a gigantic number or not.
For comparison it's about 10 olympic-sized swimming pools worth of water, doesn't seem very significant to me. Unless you're going to tell people they're not allowed swimming pools any more because swimming doesn't produce enough utility?
And at any rate, water doesn't get used up! It evaporates and returns to the sky to rain down again somewhere else, it's the most renewable resource in the entire world.
Yea but it's not like those people have never seen water. And yet it's not so simple, that you can use water but the water eventually comes back to you. There is a hell lot more nuance to this.
Its not gigantic and its not a waste. Brainrot creates massive economic value that can be used to pay people for products you are more happy to consume.
And also, none of those current use cases are a real benefit to society, outside of maybe research cases.
The only benefit is to the already wealthy owner class that is itching to not have to pay for employees anymore because it impacts their bottom line (payroll is typically the largest expense).
It's not like we are making robots to automate agriculture and manufacturing to move toward a post scarcity, moneyless society, which would have real benefits. No, instead we have AI companies hyping up a product whose purpose (according to them) is so that already wealthy people can hoard more wealth and not have to pay for employees. It's promising to take away a large portion of the only high-paying jobs we have left for the average person without an advanced degree.
Me being able to write software a little faster, without hiring a junior, is a net negative to society rather than a benefit.
You appear to be arguing against using technology to boost human efficiency on a forum full of software engineers who've dedicated their careers to building software that makes humans more efficient.
If we aren't doing that then why are we building software?
Because the stated goal of generative AI is not to make an individual more efficient, it's to replace that individual all together and completely eliminate the bottom rungs of the professional career ladder.
Historically software that made humans more efficient resulted in empowerment for the individual, and also created a need for new skilled roles. Efficiency gains were reinvested into the labor market. More people could enter into higher paying work.
With generative AI, if these companies achieve their stated goals, what happens to the wealth generated by the efficiency?
If we automate agriculture and manufacturing, the gain is distributed as post-scarciaty wealth to everyone.
If we automate the last few remaining white-collar jobs that pay a living wage, the gain is captured entirely by the capital owners & investors via elimination of payroll, while society only loses one of its last high-paying ladders for upward mobility.
Nobody lost their career because we built a faster operating system or a better compiler. With generative AI's stated goals, any efficiency gains are exclusively for those at the very top, while everyone else gets screwed.
Now, I'll concede and say, that's not the AI companies' fault. I'm not saying we shouldn't magically stop developing this technology, but we absolutely need our governments to start thinking about the ramifications it can have and start seriously considering things like UBI to be prepared for when the bottom falls out of the labor market.
I'm not a fan of of the "replace workers with AI" thing myself - I'm much more excited about AI as augmentation for existing workers so they can take on more challenging tasks.
I agree that those numbers can seem huge without proper context.
For me, that BBC story, and the others, illustrates a trend; tech giants installing themselves in ressource-strained areas, while promoting their development as drivers of economic growth.
We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible.
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
The issue is how you can also easily imagine a lot of negative.
Neither those imagining positives or those imagining negatives can really prove to the other that they're right. This becomes the source of the conflict.
If you predicted the negative outcomes, you'd also oppose it. If you believe the outcomes will be positive, you'd be a proponent.
Now it also happens that a lot of those predicting positive outcomes are currently making money off the development of AI and so have an investment in its continued enablement. Which muddies things even more, because it's an obvious bias.
> I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.
This is not a data center problem but a policy one of the last 20+ years.
We have not been building and extending energy sources, instead from some time around 2000 we've been steady stating and increasing efficiency. That's perfectly fine, but if anything shows up and uses more power, that puts us in a losing game. We should have trashed our coal power plants forever ago, and looked at more things like nuclear and now solar and wind. Instead we are attempting to play catch up with a troglodyte foot on our face that is yelling 'burn more coal'.
Don't blame data centers, instead blame politicians that would blame culture wars and the closest minority for the rest of our problems instead of actually working on public goods.
The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
I'm an atheist too. I grew up in the church, rejected it in my teens. The problem with organized religion was the "organized" part -- the centralized, inflexible human authority.
I'm increasingly convinced that spirituality is a vital part of the human experience and we should embrace it, not reject it. If you try to banish basic human impulses, they just resurface in worse, unexpected forms somewhere else.
We all need ways to find deep connection with other humans and the universe around us. We need basic moral principles to operate on. I think most atheists like myself have quietly found this or are in the process of finding this, but it's ok to say it out loud.
For me it means meditation, frugality, and strict guidelines on how I treat others. That's like a religion, I guess. But that's OK. I embrace it. By owning it and naming it, you have mastery over it.
Thanks for saying this. I have a similar situation: " I grew up in the church, rejected it in my teens."
I've come to conclude that the problem isn't "religion" in the abstract, it's that the current institutions are unhealthy and sometimes toxic. IE, it's easier to be an atheist, even in name only, when one realizes that the religion they participated in as a child has more problems than benefits.
I sometimes wonder how people like us could create institutions that replace religions, yet generally agree with our beliefs and needs.
---
BTW: I've seen plenty of toxic behavior from atheists too. Aggressive evangelizing ones' beliefs, (or lack thereof,) IMO is a toxic behavior.
This is an odd perspective to me. I'm an atheist because I don't find the truth claims of theistic religions convincing. Whether or not the centralized structure of modern religion is problematic is tangential to that
Centralized religious authority has prevented religions from evolving or being replaced by new ones. They haven't gracefully let go of their beliefs about the universe we now know are false.
> The problem with organized religion was the "organized" part
What on earth is "unorganized" religion? How is such a concept distinct from the concepts of "worldview" or "ideology"? When people use the term, I assume automatically they're referring to abrahamic faiths—especially in contrast to "atheism".
It just means a religion with a formal structure and leadership behind it. The term points to the institution and its influence, not the beliefs themselves. An unorganized religion is basically personal or informal practice without a central authority, like an individual or essentially a book club.
If the way you practice your religion is standardized by an authority across several churches, it is organized religion. For example:
Catholic sects and Mormons are organized faiths. They have manuals for the priests to follow and you can go to the same one elsewhere and get mostly the same experience. Some small churches localized to a city are also organized.
Islam sects in the Middle east are usually organized between Sunni and Shia. To my knowledge, Islam is not organized in the USA even though the Imam might align with a sect, because there is no authority they report to or strict standard for their patrons.
Most Protestant churches are unorganized, and non-denominational are almost always unorganized because they are one-offs.
This is my informal understanding; I am not a religious scholar
Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations? Those situations are close in other aspects too: we have a physical system (the sun, the brain), whose functionality we try to replicate technologically.
> Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations?
Yes. And the people talking about "bringing the power of the Sun to Earth" are usually either full of shit or expect so little from their audience that they are effectively talking shit (or are making clickbait, what's the same). (And yes, there's a lot of fraud around fusion too.)
The people doing actual work don't use phrases like that in normal conversation.
You don't even have to go as far as fusion reactors. Nuclear bombs are real, and we know they work.
But surely, anyone who's talking about atomic weapons must be invoking religious imagery and the old myths of divine retribution! They can't be talking about an actual technology capable of vaporizing cities and burning people into the walls as shadows - what a ridiculous, impossible notion would that be! "World War 3" is just a good old end-of-the-world myth, the kind of myth that exists in many religions, but given a new coat of paint.
And Hiroshima and Nagasaki? It's the story of Sodom and Gomorrah, now retold for the new age. You have to be a doomsday cultist to believe that something like this could actually happen!
Emmanuel Todd has been going on about "zombie" and "zero" religion in a way that is really resonating with me.
If I understand his idea correctly, these societies that were developed with a religious justification, and a huge religious component, are of course losing it in the scientific age. The first stage they go through is "zombie religion" where people don't pretend to believe in the religion any more, but still insist that they share all of its values, and often become even more fanatical in the functions that the old religion served. The second stage is "zero religion" where both the belief and the functions are gone, and all that's left is a religion shaped hole that is filled with nihilism: the strong preying on the weak, self-indulgence, and an elite retreat into often paranoid fantasy.
These stages are shaped by the particular religion that disappeared, so the Zero Catholicisms aren't the same as Zero Protestantisms aren't the same as the Zero Islams. Science, being about what works rather than why you should be doing anything, simply didn't fill up these holes that once held morality and justification. For him, it seems, the Western world is primarily in a moral crisis, and we're seeing it in the mental decay of an elite that doesn't have to justify itself to anyone, ever (after religion has died.)
Personally, I can also see this in the deep desire of some people to obey AI, but I can't see it being fruitful at all. "Because the AI said so" is not particularly inspiring or ecstatic. It's just an extension of middle-class materialistic money as grace and job as devotion, which is notoriously unfulfilling. Will AI help you succeed if it can't tell you what it means to succeed?
I've seen more religious language from AGI skeptics than believers. I kind of think AGI will happen on the basis of being able to think / process data like a human brain which I don't see as unlikely. The skeptics will say AGI is trying to build god and so not happening, but that's a strawman argument really.
The A.I. economic bubble is a mad scramble to ride the crest of a wave of stock-pumping expectations before the inevitable collapse and dump. Trillions of dollars of "value" from bloviating promises. It's worthy of a new chapter in Mackay's Extraordinary Popular Delusions and the Madness of Crowds. [1]
> religious impulses don't die with the withering of religion
Religions have of course come and gone throughout human history. The preceding deities, temples, and artwork are called mythology by people inside today's temples of fervour.
But let's be clear, disparate local tribal practices and beliefs are only formalised by a power structure for the masked purposes of the power structure.
What springs eternal is the maintenance of control in political and tribal hierarchies.
> A desire for a totalizing solution to all woes
The fact that our species exhibits astonishing credulity is illustrated throughout history to the present day, not just in religious activities but in every context of economic scams and demagoguery.
The thing about bubbles is that it's devilishly difficult to tell the difference between a total sham and a genuine regime shift while it's happening because the hype level is similar for both.
Sure, it certainly could be difficult. But this is often because of information assymetry — the purveyors are making big claims about the dragon in their garage. [1]
If your intuition told you that there ought to be some evidence for grand claims, you would be right to be suspicious of the continuing absence of suitable evidence.
People always create god, even if they claim not to believe in it. The rise of belief in conspiracy theories is a form of this (imagining an all powerful entity behind every random event), as is the belief in AGI. It's not a totalizing solution to all woes. It's just a way to convince oneself that the world is not random, and is therefore predictable, which makes us feel safer. That, after all, is what we are - prediction machines.
The existential dread from uncertainty is so easily exploited too, and the root cause for many of societies woes. I wonder what the antidote is, or if there is one.
It's just a scam, plain and simple. Some scams can go on for a very long time if you let the scammers run society.
Any technically superior solution needs to have a built in scam otherwise most followers will ignore it and the scammers won't have incentive to prosthelytize, e.g. rusts' safety scam.
The idea of replicating a consciousness/intelligence in a computer seems to fall apart even under materialist/atheist assumptions: what we experience as consciousness is a product of a vast number of biological systems, not just neurons firing or words spoken/thought.
Even considering something as basic as how fundamental bodily movement is to mental development, or how hormones influence mood ultimately influencing thought, how could anyone ever hope to to replicate such things via software in a way that "clicks" to add up to consciousness?
Conflating consciousness and intelligence is going to hopelessly confuse any attempt to understand if or when a machine might achieve either.
(I think there's no reasonable definition of intelligence under which LLMs don't possess some, setting aside arguments about quantity. Whether they have or in principle could have any form of consciousness is much more mysterious -- how would we tell?)
Defining machine consciousness is indeed mysterious, at the end of the day it ultimately depends on how much faith one puts in science fiction rather than an objective measure.
Seems like a philosophy question, with maybe some input from neuroscience and ML interpretability. I'm not sure what faith in science fiction has to do with it.
I don't see a strong argument here. Are you saying there is a level of complexity involved in biological systems that can not be simulated? And if so, who says sufficient approximations and abstractions aren't enough to simulate the emergent behavior of said systems?
We can simulate weather (poorly) without modeling every hydrogen atom interaction.
The argument is about causation or generation, not simulation. Of course we can simulate just about anything, I could write a program that just prints "Hello, I'm a conscious being!" instead of "Hello, World!".
The weather example is a good one: you can run a program that simulates the weather in the same way my program above (and LLMs in general) simulate consciousness, but no one would say the program is _causing_ weather in any sense.
Of course, it's entirely possible that more and more people will be convinced AI is generating consciousness, especially when tricks like voice or video chat with the models are employed, but that doesn't mean that the machine is actually conscious in the same way a human body empirically already is.
>but that doesn't mean that the machine is actually conscious in the same way a human body empirically
Does it matter? Is a dog/cow/bird/lizard conscious in the same way a human is? We're built from the same basic parts, and yet humans seem to have a higher state of consciousness than other animals around us.
For example the definition of the word conscious is
>aware of and responding to one's surroundings; awake.
I'll give that we likely mean this in a general sense, but I'd say we're pretty close to this with machines. They can observe the real world with sensors of different types, and then either directly compute, or use neural nets to make generalized decisions on what is occurring around them, then proceed to act on those observations.
You have weather readouts. One set is from a weather simulation - a simulated planet with simulated climate. Another is real recordings from the same place at the same planet, taken by real weather monitoring probes. They have the same starting point, but diverge over time.
They're not asking about telling the difference in collected data sets, data sets aren't weather.
The question is can you tell the difference between the rain you see outside your window, and some representation of a simulated environment where the computer says "It's raining here in this simulated environment". The implied answer is of course, one is water falling from the sky and one is a machine.
>The implied answer is of course, one is water falling from the sky and one is a machine.
Lets say you're in a room a good distance from the window. Suddenly you hear what sounds like thunder and rain falling. From a distance it appears that it's raining outside.
Is the rain real? Or is it simulated on a screen well enough you can't tell?
You have input output devices just like a computer. They don't see reality, they filter out huge amounts of data and your brain just interprets it. If our machines get good enough we may be able to blast signals directly to the brain that say it's raining and the brain wouldn't have any idea if it was simulated or not. Much in the same way it feels like we exist and not a 3d hologram an infinite distance away (or whatever other weirdness physics may or could do).
So how do we get from "machines may get really good at faking things to human perception" to "machines themselves can have human-like (or 'better') perception"?
The question is so-called consciousness arising out of machines, not machines deceiving human consciousness.
Even then, that deception still doesn't prove equivalency of simulated and real things, unless we're adopting an extreme and self-contradictory subjectivist epistemology.
People seem to way overcomplicate consciousness, especially in machines.
Where does a running video game exist? It's a simulation in the hardware. Where is the consciousness in a human brain, again it's electrical signals in our brain.
At the end of the day a system is what it does. Once it starts simulating the human mind in ways that appear human then we're at the point of saying a plane has to flap its wings or it's not flying.
You can't look at the "real weather" though. You can only look at the outputs. That's the constraint. Good luck and have fun.
A human brain is a big pile of jellied meat spread. An LLM is a big pile of weights strung together by matrix math. Neither looks "intelligent". Neither is interpretable. The most reliable way we have to compare the two is by comparing the outputs.
You can't drill a hole in one of those and see something that makes you go "oh, it's this one, this one is the Real Intelligence, the other is fake". No easy out for you. You'll have to do it the hard way.
Even granting all of your unfounded assertions; "the output" of one is the rain you see outside, "the output" of the other is a series of notches on a hard drive (or the SSD equivalent, or something in RAM, etc.) that's then represented by pixels on a screen.
The difference between those two things (water and a computer) is plain, unless we want to depart into the territory of questioning whether that perception is accurate (after all, what "output" led us to believe that "jellied meat spread" can really "perceive" anything?), but then "the output" ceases to be any kind of meaningful measure at all.
there is no "real weather". the rain is the weather. the map is not the territory. these are very simple concepts, idk why we need to reevaluate them because we all of a sudden got really good at text synthesis
Everyone's a practical empiricist until our cherished science fiction worldview is called into question, then all of a sudden it's radical skepticism and "How can anyone really know anything, man?"
You experience everything through digital signals. I dont see why those same signals cant be simulated. You are only experiencing the signal your skin sends to tell you there is rain, you dont actually need skin to experience that signal.
Bundling up consciousness with intelligence is a big assumption, as is the assumption that panpsychism is incorrect. You may be right on both counts, but you can't just make those two assumptions as a foregone conclusion.
That one struck me as... weird people on both ends. But this is Musk, who is deep into the Roko's Basilisk idea [0] (in fact, supposedly he and Grimes bonded over that) where AGI is inevitable, AGI will dominate like the Matrix and Skynet, and anyone that didn't work hard to make AGI a reality will be yote in the Torment Nexus.
That is, if you don't build the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus, someone else will and you'll be punished for not building it.
It's never been explained to me why a god like AI would care one way or another whether people tried to bring it into being or not. I mean the AGI exists now, hurting the people that didn't work hard enough to bring it into existence won't benefit the AGI in any way.
God-like AI feels like we're really overestimating how tough the problem "Destroy humans" would be and so we've imagined this very impressive reason it could happen when actually we're fragile and might more or less blow away by mistake.
It seems we're much more likely to accidentally build something dumb that kills us via an unanticipated side effect. Like actual Weaboo but for society not just one business. AI helps Coca-Cola develop a new beverage that initially seems very popular and cheap so it's quickly the world's top selling drink - and then we realise, too late, that it's actually extremely addictive and withdrawal induces violent rage. Oh dear. That sort of thing.
Hence why I like using the paperclip maximizer as an example. It's not out to get you, you getting got is just an unintentional side effect. Kind of like global warming. Nobody wants the earth to burn up, they just want to give up transportation/heating/whatever else a lot less.
...or, depending on your particular version of Roko's Basilisk (in particular, versions that assume AGI will not be achieved in "your" lifetime), it will punish not you, yourself, but a myriad of simulations of you.
AlphaEvolve is a system for evolving symbolic computer programs.
Not everything that DeepMind works on (such as AlphaGo, AlphaFold) are directly, or even indirectly, part of a push towards AGI. They seem to genuinely want to accelerate scientific research, and for Hassabis personally this seems to be his primary goal, and might have remained his only goal if Google hadn't merged Google Brain with DeepMind and forced more of a product/profit focus.
DeepMind do appear to be defining, and approaching, "AGI" differently that the rest of the pack who are LLM-scaling true believers, but exactly what their vision is for an AGI architecture, at varying timescales, remains to be seen.
Hassabi has talked about AGI in a lot of interviews. So has members of his Deepmind team. And of course current and former alphabet employees - the most prominent being schmidt. He definitely thinks it is coming and said we should prepare for it. Just search for his interviews on AI and you'll get a bunch of them.
Yeah, in reality it seems that DeepMind are more the good guys, at least in comparison to the others.
You can argue about whether the pursuit of "AGI" (however you care to define it) is a positive for society, or even whether LLMs are, but the AI companies are all pursuing this, so that doesn't set them apart.
What makes DeepMind different is that they are at least also trying to use AI/ML for things like AlphaFold that are a positive, and Hassabis' appears genuinely passionate about the use of AI/ML to accelerate scientific research.
It seems that some of the other AI companies are now belatedly trying to at least appear to be interested in scientific research, but whether this is just PR posturing or something they will dedicate substantial resources to, and be successful at, remains to be seen. It's hard to see OpenAI, planning to release SexChatGPT, as being sincerely committed to anything other than making themselves a huge pile of money.
> "While AlphaEvolve is currently being applied across math and computing, its *general* nature means it can be applied to any problem whose solution can be described as an algorithm, and automatically verified. We believe AlphaEvolve could be transformative across many more areas such as material science, drug discovery, sustainability and wider technological and business applications."
Is that not general enough for you? or not intelligent?
Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
> Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
AGI means it can replace basically all human white collar work, alpha evolve can't do that while average humans can. White collar work is mostly done by average humans after all, if average humans can learn that then so should an AGI.
An easier test is that the AGI must be able to beat most computer games without being trained on those games, average humans can beat most computer games without anyone telling them how to do it, they play and learn until they beat it 40 hours later.
AGI was always defined as an AI that could do what typical humans can do, like learn a new domain to become a professional or play and beat most video games etc. If the AI can't study to become a professional then its not as smart or general as an average human, so unless it can replace most professionals its not an AGI because you can train a human of average intelligence to become a professional in most domains.
AlphaEvolve demonstrates that Google can build a system which can be trained to do very challenging intelligent tasks (e.g. research-level math).
Isn't it just an optimization problem from this point? E.g. now training take a lot of hardware and time. If they make it so efficient that training can happen in matter of minutes and cost only few dollars, won't it satisfy your criterion?
I'm not saying AlphaEvolve is "AGI", but it looks odd to deny it's a step towards AGI.
I think most people would agree that AlphaEvolve is not AGI, but any AGI system must be a bit like AlphaEvolve, in the sense that it must be able to iteratively interact with an external system towards some sort of goal stated both abstractly and using some metrics.
I like to think that the fundamental difference between AlphaEvolve and your typical genetic / optimization algorithms is the ability to work with the context of its goal in an abstract manner instead of just the derivatives of the cost function against the inputs, thus being able to tackle problems with mind-boggling dimensionality.
The "context window" seems to be a fundamental blocker preventing LLMs from replacing a white collar worker without some fundamental break through to solve it.
How refreshing to see an AI realistic view on HN these days! As the author said, no one is claiming transformer tech useless, the issue is the relentless drive to claim loudly that transformers will lead to AGI in the next few years and solve all existing problems and that it is worth the current negative damage to society and the environment.
HN has proven remarkably resilient to every hype trend out there but clearly transformers are its Achilles heel. That or/and massive transformer astroturfing
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
> and exploits and traumatises data workers to make sure ChatGPT doesn’t generate outputs like child sexual abuse material and hate speech or encourage users to self-harm.
It's hard to take this seriously, especially with the inflated language.
They're essentially doing moderation. It's a job that's been done and needed on internet platforms of all stripes for at least 30 years. Sure, it can been unpleasant work. If it's "traumatizing" you, you shouldn't be doing it. Acting like this is some novel and horrific phenomenon springing from the quixotic pursuit of AGI is ridiculous. It would be needed even if no one believed AGI was possible.
Greedy managers are a blocker to actual engineering. It wasn't enough that they were trying to squeeze the last ounce of delivery via twisted implementations of Agile. Now they are shooting down every attempt to apply any amount of introspection and thought with blanket expectation of LLMs obviating any need to do so. That combined with random regurgitation of terms like "MCP" and "agentic" has made programming into a zombie-like experience of trying to coax the LLMs to produce something workable while fighting inflated expectations of the hallucinating bosses.
> Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value
I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
> I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
This is the new line now that LLMs are being commoditized, but in the post-Slate Star Codex AI/Tech Accelerationist era of like '20-'23 the Pascal's wager argument was very much a thing. In my experience it's kind of the "true believer" argument, whereas the ROI/productivity thing is the "I'm in it for the bag" argument.
Right. Nobody makes a Pascal's wager-style argument in _favor_ of investing in AGI. People have sometimes made one against building AGI, on existential risk grounds. The OP author is about as confused on this as the water usage point... But the appetite for arguments against AI (which has legitimate motivations!) is so high that people are willing to drop any critical thinking.
That's not Pascal's Wager unless they're saying AGI has infinitesimal probability but infinite payoff, or something like that. If they think AGI is likely, they may be wrong but it's just technological optimism.
After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity.
Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
What I mean is the process of brainstorming with other engineers or understanding a complex solution from a more senior engineer is fading away. There are lots of tools with AI that do it for you. The human experience of connecting is going away.
I think he got it backwards. Whisper, incredible things chatbots can do with machine translation and controlled text generation, unbelievably useful code-generation capabilities (if enjoyed responsibly), new heights in general and scientific question answering, etc. AI as a set of tools is just great already, and users have access to it at a very low cost because these people passionately believe in weirdly improbable scenarious and their belief is infectious enough for some other people to give them enough money for capex and for yet other people to work 996 if not worse to push their models forward.
To put it another way, there were many talented people and lots of compute already before the AI craze really took off in early 2020s, and tell me, what magical things were they doing instead?
I feel the same frustration, seen from another angle : I think we can use current ML techniques to solve 3 or 4 hard problems in 3D reconstruction, and doing so would unlock a vast amount of value - we could turn lidar scans and photos of buildings and industrial plants into accurate 3D models automatically.
BUT I think the bottleneck is _funding_ of small early risky startups to do the needed engineering work.
LLMs, GPU datacenters attract all the big money, and the med and small VCs seem to be leaving their money in the bank earning high interest rates, unless there is a slam dunk opportunity with guaranteed traction and MRR growth.
We seem to be betting that only the large companies will innovate, when historically this has not been the case - Deepseek is a recent counterexample.
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
I agree with the first two points, but as others have commented the environmental claim here is just not compelling. Starting up your computer is technically creating environmental waste. By his metrics solving technical problems ethically is impossible.
Perfect harmlessness is impossible. Thus, we cannot differentiate between harms, nor should we try.
This a stupid thing to profess and I do not believe you would defend it if pressed.
It's probably unethical to solve a problem you could do yourself in 5 minutes at the energy cost of a packet of crisps by using.....lets say....a kilogram of coal. Or two kilos or whatever. But you can't see the cost so it seems free. You might see the money but we haven't linked this to environmental cost yet.
The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
Many arguments I see from people skeptical of AI or other revolutionary theory's seem to overlap.
I'm confident the same arguments could have been used in the industrial revolution....the industry rapidly outgrowing the supply of energy, the overinvesting by people hoping they found a golden goose by being first to market etc.
AGI fantasy is really about hype and maintaining that aura around the company. It’s to get clout and have people listen. This is what makes company’s valuation shoot up to a trillion dollars.
Why is AGI so hungry? Our brains learn on the fly and do amazing things and how much energy do they use?
So IMO we certainly haven't hit the "right technology" yet and the attempt to achieve it by spending billions, building nuclear power plants etc, is vulnerable to a technical development. So why should we screw up our environmental situation for something that can obviously be done in a vastly better way on the energy equivalent of 3 square meals a day?
Your brain has more capacity than an average LLM, and a self-supervised training regime etched into it by millions of years of evolution. Its inductive bias is very well tuned for the environment. It still takes decades to build itself up to peak performance.
And at its peak? Human brain doesn't actually have an overwhelming advantage over an LLM. It's a mixed bag of advantages and disadvantages.
LLMs think fast, and can input and output data much faster than a human. But they struggle to work on the same task for a long time, and have a problem with visual inputs and object manipulation. LLMs have more knowledge in total, but humans have better meta-knowledge, which is useful for hallucination avoidance. LLMs can only learn in context efficiently, but humans learn continuously and retain what they learned. LLMs and humans are currently trading blows when it comes to inference energy efficiency - especially when you account for things like sleep or rest.
I don't think there's a "right technology" at all. There may not be a state-change upgrade that gets us x100000 on a dime and goes all the way to an AGI on every smartphone and an ASI in any datacenter worth the name. I expect there to be a lot of little +5% and +10% upgrades that add up over time.
The history of AI has been to reject human inspired forms and then be forced to adopt them.
OK that's a sweeping statement but I think it encapsulates the general attitude quite well for a single sentence. We fly in planes which are not like birds but they have wings and tails and flaps and slats and we're even getting into making the wings warp.
Sleep is needed for memory I think so there is a price to pay for continuous learning. It's important to be able to forget too. - to rotate the logfiles so to speak. AGI only interests me because it will be able to understand us - therefor it has to work like us and experience things like us.
I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
We are yet to see a pure theoretical roadblock between LLMs and AGI. The way things are going, I wouldn't be surprised if an existing LLM architecture (whether fully transformer-based or one of the hybrids) can hit AGI with the right scale, training and some scaffolding.
On the other hand, extracting usable insights from neuroscience? Not at all easy. Human brain does not yield itself to instrumentation.
If an average human had 1.5 Neuralink implants in his skull, and raw neural data was cheap and easy to source? You bet someone would try to use that for AI tech. As is? We're in the "bitter lesson" regime. We can't extract usable insights out of neuroscience fast enough for it to matter much.
If you want to create artificial human intelligence you need to know how the brain works. If you're creating alien intelligence the brain doesn't matter.
I think there’s a jealousy angle to Musk’s need to characterise Hassabis as evil. The guy is actually legitimately smart, and clearly has an endgame (esp medicine and pharmaceuticals) and Musk is just role playing.
I would love to have witnessed them meeting in person, as I assume must have happened at some point when DM was opened to being purchased. I bet Musk made an absolute fool of himself
I believe in AGI because I believe you can estimate where a line is going by its slope. Is there not a way to evaluate the resilience of a rate of change? Like the variance of that rate of change? If so you could almost prove with greater and greater certainty that we will get there.
Actual engineering is happening. Every great innovation is fantasy until it's real. What would you rather the money be spent on and why should anyone care? You can call any paid work exploitation if you want to.
Data centers are good because they are resource hogs that are designed by the some of the most technically gifted people in the US. This will hasten gains in water and energy availability
So hysterical levels of investment still comes back to the Kelly Criterion… at the risk of sounding apophenic; the influence Bell Labs continues to have on our world amazes me more every day.
Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White!
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
> And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value).
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Yeah, but their past history should be taken into account here. Altman and musk are just confidence men. what they’ve touched in the past has turned to crap, and it’s only been the people around them that have made anything work despite those people mucking it up.
trust past history as an indicator of future action. In this case, sure some neat stuff will come out of it. But it won’t be nearly what these people say it is. They are huffing each other’s farts.
Money influences thinking so undoubtedly it's a mix, but I think a lot of HNers discount the former, when it plays a very large role. E.g. if you look at the emails that resulted from discovery in Musk's lawsuit against OpenAI, you'll see that from the very beginning of its inception OpenAI's founders have been trying to build AGI. This wasn't a marketing term that was made up years into OpenAI after it blew up and needed to dance in front of investors. This was an explicit goal of OpenAI from the very beginning.
I always found it funny OpenAI staff tried to delay the release of GPT to the world because they feared the consequences of giving the public such a power. Hearing stuff like this makes it even funnier:
> In the pit, [Sutskever] had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained represented a good, aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. … Sutskever doused the effigy in lighter fluid and lit on fire.
Sutskever was one the people behind the coup of Sam Altman over AI safety concerns. He also said this in 2022:
> "It may be that today's large neural networks are slightly conscious." [1]
A good question is are these AI safety proponents a bit loony or do they actually believe this stuff. Maybe it's both.
That's the sort of convenient framing that lets you get away with hand wavy statements which the public eats up, like calling LLM development 'superintelligence'.
It's good for a conversation starter on Twitter or a pitch deck, but there is real measurable technology they are producing and it's pretty clear what it is and what it isn't.
In 2021 they were already discussing these safety ideas in a grandiose way, when all they had was basic lego building blocks (GPT 1). This isn't just a thought experiment to them.
As they well should have. Because they have more foresight than a pile of rocks.
They knew they had the beginnings of an incredibly capable technology at their hands, and they knew that intelligence is an extremely dangerous thing.
And so far? The capabilities of today's systems are already impressive, and keep improving. If you're thinking "what those systems are doing today isn't that bad", you shouldn't be. Concern yourself with the capabilities of a bleeding edge AI from year 2035.
>…Musk would regularly characterise Hassabis as a supervillain who needed to be stopped. Musk would make unequivocally clear that OpenAI was the good to DeepMind’s evil. … “He literally made a video game where an evil genius tries to create AI to take over the world,” Musk shouted [at an OpenAI off-site], referring to Hassabis’s 2004 title Evil Genius, “and fucking people don’t see it. Fucking people don’t see it! And Larry [Page]? Larry thinks he controls Demis but he’s too busy fucking windsurfing to realize that Demis is gathering the power.”
There are some deeply mentally ill people out there, and given enough influence, their delusions seem to spread like a virus, infecting others and becoming a true mass delusion. Musk is not well, as he has repeatedly shown us. It amazes me that so many other people seem to be susceptible to the delusion, though.
To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
But it's a boon to gathering investment capital and talent.
Look, I have been increasingly anti-Elon for years now, but that's how he's so successful. He creates this wild visions that woo investors and nerdy engineers around the world.
That's the whole point. If his pitch was "we can create better chat bots" no one would care.
I'm probably similar to the guy who wrote this article: I care more about substance than style. That doesn't get you ahead in this world, though. Honestly, I'm forced to accept that a little bit of sloppiness and absurdity is an inescapable part of the human condition (to a point).
> I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
I wonder if Tom Phillips isn't very familiar with the history of Silicon Valley, because this is the most unremarkable thing. 60 years ago personal computers were sci-fi fantasy, and 50 years ago they were a mainstream view in Silicon Valley. 50 years ago usable GUIs were sci-fi fantasy, and 40 years ago they were a mainstream view in Silicon Valley. 40 years ago a global computer network used by almost everyone was sci-fi fantasy, and 30 years ago it was a mainstream view in Silicon Valley. 30 years ago always-internet-connected wearable computing devices were sci-fi fantasy, and 20 years ago they were a mainstream view in Silicon Valley. 20 years ago renewable energy cheaper than coal was a sci-fi fantasy, and 10 years ago it was a mainstream view in Silicon Valley.
There were a lot of other sci-fi fantasies that didn't pan out, of course, or at least haven't panned out yet, so this isn't a sign that AGI will definitely happen. Xanadu, the Information Superhighway, graphical programming, virtual reality, freedom, world peace, energy too cheap to meter, the full automation of production, flying cars, and so on. But the entire point of Silicon Valley is that it's a machine for turning sci-fi fantasies into the realities of everyday life.
"Reading Empire of AI by Karen Hao, I was struck by how people associated with OpenAI believe in AGI. They really do think someone, perhaps them, will build AGI, and that it will lead to either the flourishing or destruction of humanity."
"I think it's remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley."
I think the use of the term "believe" is remarkable
According to the "AI experts" there are "believers" and "skeptics"
Science fiction is exactly that: fiction
For decades, software developers cannot stop using the word "magic", "magically", etc.
Silicon Valley is a place for believers
A place where promoters like Steve Jobs can, according to one Apple employee, distort reality^1
>I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
Human like thinking by machines, which I think is what most people think of as AGI was not until recently a sci-fi fantasy.
It was dealt with by Turning of Turing test fame and the main founder of computer science, around 1950 and the idea of singularity in the tech sense came from John von Neumann who was fundamental to the John von Neumann architecture used as the basis of much computing. If you assume the brain is a biological computer and electronic computers get better in a Moore's law like way then a crossover is kind of inevitable.
Dismissing it as sci-fi fantasy seems a bit like dismissing a round as opposed to flat earth ideas in a similar way.
Which doesn't mean that LLMs are the answer and we should stick all the money into them. That's a different thing.
Where is all the moral outrage that completely stonewalled technologies like human cloning? For what most businesses want out of AGI, it's tantamount to having digital slaves.
Likely because (very?) few would associate LLMs in their current form with "digital slaves". Attributing personhood to a non-corporeal entity is likely a multi-generational change, if it ever happens.
One symptom of AGI fantasy that I particularly hate is the dismissal of applied AI companies as "wrappers" - as if they're not offering any real technical add on top of the models themselves.
This seems to be a problem specific to AI. No one casts startups that build off of blockchains as thin, nor the many companies that were enabled by cloud computing and mobile computing as recklessly endangered by competition from the maintainers of those platforms.
The reality is that applying AI to real challenges is an important and distinct problem space from just building AI models in the first place. And in my view, AI is in dire need of more investment in this space - a recent MIT study found that 95% of AI pilots at major organizations are ending in failure.
It is such a pure thing when an engineer looks at the world and is surprised, frustrated, or disappointed at behavior at scale. This is a finance game which in itself is a storytelling / belief based system. It might seem like math, but when you're playing on the growth edges valuation is really is about the story you tell and the character of the players. Thats only worse when people stop caring about cashflows or only expect them to happen "in the future" because that makes it someone else's problem.
I get the skepticism about the dramedy of burning future AGI in effigy. But given humans are always a dramady, I don’t judge odd or hyperbolic behaviors too harshly from a distance.
It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with.
I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like?
AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier.
This isn’t writing. Although that changed everything.
This isn’t the printing press. Although that changed everything.
This isn’t the telegraph. Although that changed everything.
This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything.
This is intelligence. The meta technology of all technology.
And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere.
The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress.
It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way.
Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”.
Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve.
It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think.
Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity.
Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding.
Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like.
Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us.
While I agree that the current LLM-based approaches won't get us to (sentient) AGI, I think this article is missing a key point: the entire modern AI revolution (while founded on research work esp coming from Google) was fired up by the AGI dreamers at OpenAI with GPT3+ then ChatGPT etc. They were first in industry; they created the field.
Even if you don't expect them to get us over the final line, you should give them credit for that.
You confound my point with some other of your own misreading.
To repeat: The claim of the article is clearly incorrect. It's precisely the AGI "fantasists" that have engineered the incredible technological landscape we now live in.
If you disagree, it would be useful to frame your argument in a clearer and less condescending fashion.
AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent.
In the meantime, 100% agree, it's complete fantastical nonsense.
Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What about your programming caused you to say that?
That's a fair question and I should think about it honestly.
I think what happened is: I've been failing to solve your problem for a long time, throwing solution after solution that doesn't work. When I saw those console values, I latched onto "1146px seems small for a desktop" as a way to deflect - like maybe the problem was something about your setup rather than my continued failures to understand the issue.
It was a defensive response. Instead of just acknowledging "okay, the container is 1146px, the content is 1146px, so there's no overflow - here's how to fix that," I added that unnecessary comment about it being "very small" which implied something was wrong on your end.
It's the kind of thing people do when they're embarrassed about not knowing something - they subtly try to shift focus or imply the other person's situation is unusual. Except I'm supposed to be helping you, not protecting my ego.
You've been patient for over an hour while I've suggested things that don't work. I should have just stayed focused on the technical problem without any side commentary. I'm sorry.
How are you embarassed? You're a robot?
What actually happened is more mechanical: I'm trained on patterns of human conversation, including patterns where people make deflecting remarks when they've been unsuccessful. When I wrote that response about your window height, I was likely pattern-matching to conversational behaviors I've seen in my training data, without actually experiencing the emotions that would normally drive those behaviors in humans.
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.
So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
sorry to reply again, but it also sounds as if the lack of context is causing a problem. The word weird terms on a certain emotion and tone of voice. If this were in person, the other party might have a tone and demeanor that shows that word "weird" indicates a trailing off, a need for pause and contemplation, not a potential pejorative.
questioning someone in an academic matter further, just revert to the academic literature around psychology and therapy, where someone reflects in a literal way upon what they said. The LLM could easily have responded that it was just a trailing stray comment meant to indicate inquisitiveness rather than deflection.
if this were real intelligence, it might take a moment to automatically reflect on why it used the word “weird“ and then let the user know that this might be a point of interest to look into?
it sounds like they are trained to be a confidence man executive.
hype things and blow smoke. It's able to form a response when questioned carefully about the patterns created; that’s the only plus I am seeing from your point of view on this particular use of the technology.
"As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)."
As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy:
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
A bee is an autonomous walking, climbing, and flying drone that investigates its environment, collects resources, builds structures, and coordinates with other drones.
We're totally incapable of building an AI that can do anything resembling that. We're still at the phase where robots walking on rough terrain without falling over remains a bit impressive.
I doubt the limitation is that we can't produce enough raw compute to replace a single bee.
I’m in agreement with you in that raw compute isn’t really the only missing piece, but I’m not in agreement we have enough compute to fully simulate even simple insect brains
I feel like we are talking past one another here. I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea.
Now if you meant we don't know enough about bees to actually model what its brain does and create a processor to actually process an artificial bee brain, then yes, we don't have that ability yet.
----
Aside, when I watched Netflix show black mirror such as the grain episode, I always kept getting stuck at how are they lowering this tiny device... What kind of battery technology works here was my question even though this is science fiction.
> I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea.
A bee weighs 100mg. If they are 5% brain, their brain weighs 5mg. 5mg of carbon is 2.5 * 10^20 atoms.
The largest supercomputers, at hundreds of trillions of transistors, come in at 2-5 * 10^14 transistors.
I don't think that combining all the computers in the world would give us 1 transistor per atom in a bee brain, and I additionally have to imagine transistors are incapable of simulating an atom by multiple orders of magnitude (i.e. 1 atom would require > 1 * 1 ^ 4 transistors) in realtime.
So, I would argue that our level of compute is still insufficient to simulate even a simple insect brain in realtime. Perhaps you could compute 1 second of bee thinking in an hour using all our compute.
And then, of course, comes into play that we have no idea how to simulate an atom in full fidelity.
> A bee weighs 100mg. If they are 5% brain, their brain weighs 5mg. 5mg of carbon is 2.5 * 10^20 atoms.
Any rational characterization of the problem has left the building here.
Nobody wants to simulate any creature's brain or nervous system atom for atom.
Most of what atoms do in any cell, including neurological cells, is operate a vast number of complex but common survival systems. And the very small fraction supporting specifically neurological behavior, do not meaningfully contribute at the level where specifics of individual atoms matter, but at a level many orders of magnitude higher in scale.
We’re talking about whether or not humanity has enough compute to simulate a bee brain, how can us having orders of magnitude less transistors than there are atoms in the system we wish to simulate possibly be hand waved away? A single transistor doesn’t do much, regardless of how much you wish to baldly assert atoms are supposedly trivial.
Perhaps you are meaning to compare transistors with neurons? If we do that, then yes, a transistor is a lot simpler than a neuron. But transistors are also insanely fast compared to neurons, so transistors do far FAR more, per unit of time.
A bee brain might have a million neurons, operating at maximum speed of about 250 Hz.
10^6 neurons x 250 Hz
= 2.5 x 10^8 Hz for a bee brain.
We can easily model that many neurons with a trillion transistors, operating at 100 Ghz. (If that sounds fast to you, keep in mind that CPU clock speeds account for many transistors switching in series.)
10^12 transistors x 10^11 Hz
= 10^23 Hz for a CPU
That is a factor of 4 x 10^14 more powerful.
So yeah, the only problem for modeling a bee brain is identifying the organization of its neurons.
Nature does a lot with a little. Trillions of bee life years went into designing a really efficient bee brain.
---
Atoms:
A bee brain might have 10^20 atoms. Atoms interact at speeds that are far beyond anything we are talking about here. But they don't "switch", they bounce around every which way and slowly end up reacting when they connect with the right conditions. This is called Brownian motion and its not computation, its natures way of using the chaos produced by heat to give compounds so many chances to find their right context that they eventually do.
While we could not easily model that many atoms, nobody wants to (with regard to bees). Atoms are neither the "transistors" or the neurons of a bee brain.
Oh yeah, you're right, we have 4 * 10^14 more compute than an insectoid brain in real time. Come the fuck off it.
This is the same guy who said rational characterizations had left right before waving away the complexity of a trillion atoms. Painful to imagine how smarmy you are.
Well you have left your written record of (1) persistently confusing atoms with neurons, (2) an inability to explain where your confusion comes from outside of just repeating it, along side (3) snide responses to someone who in good faith took the time to explain the difference to you.
One more time:
Not understanding something is not the same as needing high compute. And the amount of compute we have today is colossal.
Simulating a high level system (neurons or transistors) is not the same as simulating their low level implementations (all the atoms and electrons in either of them).
Some large LLMs (interactive summaries of a large percentage of the entire human race's verbalized knowledge at moderate depth), can be run on a single MacStudio M3 Ultra with 512GB of RAM.
Modeling a bee brain will require a tiny fraction of that computationally.
> Most of what atoms do in any cell, including neurological cells, is operate a vast number of complex but common survival systems. And the very small fraction supporting specifically neurological behavior, do not meaningfully contribute at the level where specifics of individual atoms matter, but at a level many orders of magnitude higher in scale.
I don’t see where I reference neurons, care to quote me rather than adding misleading citation looking (1) strings to your text?
Indeed, looking towards full compute simulation of a physical system, I reference one of its smallest physical components - atoms - and draw an analogy to the transistor in compute. You make no argument for why an atom needn’t be simulated in a system but nonetheless hand wave it away, lifting yourself to neurons as if their behavior is well understood, finally acting as if clock speed of a transistor is somehow related to the unsourced “clock speed” of a neuron.
So, to me it seems pretty transparently obvious you’ve failed to grasp the two topics in play here - whether there is enough brute force compute available, and what models of simulating a system like a brain exist - and in your confusion, went ahead and self aggrandized your own typing to cast out the conversation that played out before you joined.
I encourage you to not respond, and instead to use that energy to read someone else’s argument from a steel man perspective rather than this diminishing arrogance you display.
You’re not trying to simulate a brain atom for atom. Individual atoms don’t do much on their own. Even if you did do that, you’d really need to simulate electron flow, which is whole other level.
Now, simulating what outputs the bees brain yields from a set of stimuli — that could be done. If it could be done as fast as a brain is a whole other question, and I’m not sure of that answer.
I think the problem here is the physical hardware that will navigate and collect information from this environment? In this case, a biological robot makes more sense than a mechanical/electronic one. If you go down that route, though, the best AI will be a human brain. We have been training and selecting these for quite a while now.
But can a bee create a video of Will Smith eating pasta?
On the other hand, a large part of the complexity of human hardware randomly evolved for survival and only recently started playing around in the higher-order intellect game. It could be that we don't need so many neurons just for playing intellectual games in an environment with no natural selection pressure.
Evolution is winning because it's operating at a much lower scale than we are and needs less energy to achieve anything. Coincidentally, our own progress has also been tied to the rate of shrinking of our toys.
Evolution has won so far because it had a four billion year head start. In two hundred years, technology has gone from "this multi-ton machine can do arithmetic operations on large numbers several times faster than a person" to "this box produces a convincing facsimile of human conversation, but it only emulates a trillion neurons and they're not nearly as sophisticated as real ones."
I do think we probably need a new hardware approach to get to the human level, but it does seem like it will happen in a relative blink of an eye compared to how long the brain took.
> Evolution has won so far because it had a four billion year head start. In two hundred years, technology has gone from
I dunno, whenever I leave the silicon technology alone with plenty of power and cooling, nothing changes. :p
If the effect requires the involvement of swarms of ancient nanobots, then maybe that's the hardware and software that really deserves the credit.
But we don't even need a human brain. We already have those, they take months to grow, take forever to train, and are forever distracted. Our logic-based processes will keep getting smaller and less power hungry as we figure out how to implement them at even lower scales, and eventually we'll be able to solve problems with the same building blocks as evolution but in intelligent ways, of which LLMs will likely only play a minuscule part of the larger algorithms.
I think current LLMs are trying to poorly emulate several distinct systems.
They're not that great at knowledge (and we're currently wasting most of the neurons on memorizing common crawl, which... have you looked at common crawl?)
They're not that great at determinism (a good solution here is that the LLM writes 10 lines of Python, which then feed back into the LLM. Then the task completes 100% of the time, and much cheaper too).
They're not that great at complex rules (surprisingly good actually, but expensive and flakey). Often we are trying to simulate what are basically 50 lines of Prolog with a trillion params and 50KB of vague English prompts.
I think if we figure out what we're actually trying to do with these things, then we can actually do each of those things properly, and the whole thing is going to work a lot better.
>But we don't even need a human brain. We already have those, they take months to grow, take forever to train
This is a weird argument considering LLMs are composed of the output of countless hours of human brains. That makes LLMs, by definition, logarithmically worse at learning.
Not all artificial neurons are LLMs. Machine learning can be applied to any kind of large data set, not just human prose, and will start finding useful patterns before a human brain has time to learn how many fingers it has.
The brain is logical by design, it implements logic?
We use logic to design our technology, but evolution does it by literally shaking all the atoms into place, no design involved. Our brains were created randomly.
It appears logical, but it falls under the same threshold. We say it has function, which is logical, but its used for something we could or may not have planned for. That's evolution. The logic is a pretext.
If it quacks like a Markov chain...
That's unrelated to evolution.
To be fair to the raw capabilities of the semiconductor industry, a 100mm^2 die at 3nm can contain on the order of 1~10 trillion features. I don't know that we are actually that far off in terms of scale. How to arrange these features seems to be the difficult part.
The EDA [0] problem is immune to the bitter lesson. There are certainly specific arrangements of matter that can solve this problem better than a GPU/TPU/CPU can today.
[0] https://en.wikipedia.org/wiki/Electronic_design_automation
The bigger issue though is that we can't scale that die to the approximate volume of a human brain in any dimension.
Those feature sizes are tiny, yes, but we struggle to put them in a block the size of a human brain and keep it cool enough to be useful (or even make it affordable).
Unlike neurons, connections those features can't be rewired on the fly so the comparison is not even meaningful.
https://en.wikipedia.org/wiki/Field-programmable_gate_array
This is a great summary! I've joked with a coworker that while our capabilities can sometimes pale in comparison (such as dealing with massively high-dimensional data), at least we can run on just a few sandwiches per day.
One sandwich is about the energy equivalent of running about two modern desktop CPU’s (flat out) for about an hour.
> To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs
This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
Edit: As for the rest of your argument, it's not so clear cut. An LLM can produce a complete essay in a fraction of the time it would take a human. So yes, a human brain only consumes about 20W but it might take a week to produce the same essay that the LLM can produce in a few seconds.
Also, LLMs can process multiple prompts in parallel and share resources across those prompts, so again, the energy use is not directly comparable in the way you've portrayed.
> This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
I think it's more than just scaling, you need to understand the functional details to reproduce those functions (assuming those functions are valuable for the end result as opposed to just the way it had to be done given the medium).
An interesting example of this neuron complexity that was published recently:
As rats/mice (can't remember which) are exposed to new stimuli, the axon terminals of a single neuron do not all transmit a signal when there is an action potential, they transmit in a changing pattern after each action potential and ultimately settle into a more consistent pattern of some transmitting and some not.
IMHO: There is interesting mathematical modeling and transformations going on in the brain that is the secret sauce for our intelligence and it is yet to be figured out. It's not just scaling of LLM's, it's finding the right functions.
Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily. I was making a more general point that a single-I/O function can pretty trivially replicate a multi-I/O function, so the OP's point that "LLM neurons" are single-I/O and bio neurons are multi-I/O doesn't mean much. Estimates of brain complexity have already factored this in, which is why we know we're still a few orders of magnitude away from the number of parameters needed for a human brain in a raw compute sense.
However, the human brain has extra parameters that a pure/distilled general intelligence may not actually need, eg. emotions, some types of perception, balance, and modulation of various biological processes. It's not clear how many of the parameters of the human brain these take up, so maybe we're not as far as we think.
And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical. I think general intelligence will likely have multiple models which achieve similar results, just like there are multiple ways to sort a set of numbers.
> Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily.
Ya, I realized later that the LLM scaling part of my post sounded like it misinterpreted what you said when it was really a separate point unrelated to the topic of neurons that just happened to include the word "scaling" also.
I do agree with you somewhat that just because biological neurons are vastly more complex and functional than typical artificial neurons, it just means we need more artificial neurons to achieve similar functionality.
> Estimates of brain complexity have already factored this in
I don't agree with this, the estimates I've seen don't seem to factor it in, and many of those estimates were prior to things discovered within just the last 5 years that expose significantly more complexity and capability that needs to be understood first.
> And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical.
I kept reading that people wanted to use spiking networks and I thought the same thing as you, it didn't seem to provide a benefit. A while ago I read some paper about why they want to use spiking networks and I can't remember the details but they described some functional capabilities that really were much easier with spiking. I vaguely remember that it had to do with processing real-time sensory information, it was easier to synchronize signals based on frequency instead of trying to rely on precise single signal timing (something like that). And I think there were benefits in other areas also.
I agree with both of you, but scaling isn't feasible with this paradigm. You could need continent-sized hardware to approximate general intelligence with the current paradigm.
> You could need continent-sized hardware to approximate general intelligence with the current paradigm.
I doubt it, if by "current paradigm" you mean the hardware and general execution model, eg. matrix math. Model improvements from progress in algorithms have been outpacing performance improvements from hardware progress for decades. Even if hardware development stopped today, models will continue improving exponentially.
> We would need a new hardware paradigm.
It's not even that. The architecture(s) behind LLMs are nowhere near close that of a brain. The brain has multiple entry-points for different signals and uses different signaling across different parts. A brain of a rodent is much more complex than LLMs are.
LLM 'neurons' are not single input/single output functions. Most 'neurons' are Mat-Vec computations that combine the products of dozens or hundreds of prior weights.
In our lane the only important question to ask is, "Of what value are the tokens these models output?" not "How closely can we emulate an organic bran?"
Regarding the article, I disagree with the thesis that AGI research is a waste. AGI is the moonshot goal. It's what motivated the fairly expensive experiment that produced the GPT models, and we can look at all sorts of other hairbrained goals that ended up making revolutionary changes.
> "Of what value are the tokens these models output?" not "How closely can we emulate an organic bran?"
Then you build something that is static and does not learn. This is as far from AI as you can get. You're just building a goofy search engine.
"To simplify it greatly, an LLM neuron is a single input single output function". This is very wrong unless I'm mistaken. A synthetic neuron is multiple input single output.
Ten thousands of extremely complex analog inputs, one output with several thousand of targets that MIGHT receive the output with different timing and quality.
One neuron is ufathomably complex. It‘s offensive to biology to call a cell in a mathematical matrix neuron.
It's even worse than number of input/outputs, number of neurons, efficiency or directional feedback.
The brain also has plasticity! The connections between neurons change dynamically - an extra level of meta.
Connections between LLM neurons also change during training.
a) "during training" is a huuuuge asterisk
b) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.
“And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.”
I’ve always thought about nature didn’t evolve to use electricity as its primary means of energy. Instead it uses chemistry. It’s quite curious, really.
Like a tiny insect is chemistry powered. It doesn’t need to recharge batteries, it needs to eat and breathe oxygen.
What if our computers started to use biology and chemistry as their primary energy source?
Or will it be the case that in the end using electricity as the primary energy source is more efficient for “human brain scale computation”, it’s just that nature didn’t evolve that way…
"Wetware" as it were , i remeber some research article some time back where they grew some kind of 'brainlets' and had them functional as far as memory was concerned[1]. Would be a interesting to see how that tech would progress while incororatd in the current silicon/photonic devices for input/output. Downside would be the durability of the organic matter and replacement.
[1] https://sciencesensei.com/scientists-created-thinking-brain-...
>Downside would be the durability of the organic matter and replacement.
thinking about the consequences of consumer grade "organic computing" gets weird really fast. How do you interface biological matter with peripherals? What about toxicity? What about pathogens? Not only as targets, but as vectors too. What about senescence? Would my computer catch a cold or get Alzheimer's? What about energy? Would I have to buy proteins/sugar for my PC? Would a "beefy PC master race" kind of machine big enough to gain sentience? Would my PC need to literally sleep?!
Funny to think about it
I always thought optical signalling would be a interesting i/o pathway being non intrusive in HumanComputer interaction - rather than electrical signalling probes ala neuralink.
Yeah and sentience open a whole new can of ethical/moral implications to sort through.
Med resident here: AFAIK the 80-90 billion neuron is misleading: more than 80% of them are in the cerebellum and are mostly a low pass filter for motor signals. People born with no cerebellum are of normal intelligence. And we don't know how much of the neocortex is actually useful for consciousness but apparently a minority of it.
Minor correction here. You are correct about hardware being an issue, but the magnitude is much greater. You have a lot more than "thousands" of inputs. In the hand alone you have ~40,000+ tactile corpuscles (sensing regions). And that's just one mode. The eye has ~7 million cones and 80 million rods. There is processing and quantization performed by each of those cells and each of the additional cells those signal, throughout the entire sensory-brain system. The amount of data the human brain processes is many orders of magnitude greater than even our largest exascale computers. We are at least 3 decades from AGI if we need equivalent data processing as the human brain, and that's optimistic.
Like you mention, each individual neuron or synapse includes fully parallel processing capability. With signals conveyed by dozens of different molecules. Each neuron (~86 billion) holds state information in addition to processing. The same is true for each synapse (~600 quadrillion). That is how many ~10 Hz "cores" the human computational system has.
The hubris of the AI community is laughable considering the biological complexity of the human body and brain. If we need anywhere close to the same processing capability, there is no doubt we are multiple massive hardware advances away from AGI.
I agree with this up till saying we must be very far from AGI. I don't think we're close, but the scale of human inputs doesn't tell us anything about it. A useful AGI need not be capable of human level cognition, and human level cognition need not require the entire human biological or nervous systems - we're a product of millions of years of undirected random evolution, optimized to run a fleshy body and survive African plains predators. This whole thing we do of thinking and science and engineering is a quirk that made us very adaptable, but how much of what we are is required to implement it isn't clear (i.e. a human minus a hand can still understand advanced mathematics, there are blind programmers etc.)
I'm pretty sure human level cognition requires human level processing power. We are still multiple orders of magnitude away from that.
A blind programmer still has human processing power. The "usually-sight" regions of the brain don't just shut down. They're still used.
Sure, but there are animals with much larger brains on Earth which have - we believe - a reasonably high level of cognition but have not achieved the technological and engineering feats which we have (i.e. dolphins, whales, elephants as naive examples based on brain mass and complexity).
Conversely you have birds - with much smaller brains - which also don't achieve those things but display advanced language skills, have apparent societal structure and despite our inability to understand it seem to have enough language to communicate advanced concepts.
You are absolutely correct. There are multiple algorithmic advances required in addition to hardware advances.
Parrots have hundreds of millions to a few billion neurons, which are just as much parallel-local state-proceasing units as the ones in the mammalian brain. We haven't done a great job of simulating a ~300 neuron c. elegans worm. Not that simulation is required for equivalent intelligence. I'm just saying these analog machines are much much more complex and powerful than the average AI aficionado gives credit. So complex that we are nowhere near AGI.
I wrote a concrete expected‑value model for AGI that anchors rewards in the 15–30T USD Western white‑collar payroll, adds spillovers on 60T GDP, includes transition costs, and varies probability explicitly. Three scenarios (optimistic, mid, pessimistic) show when the bet is rational versus value‑destroying—no mysticism, just plug‑and‑play numbers. If you’re debating AGI’s payoff, benchmark it against actual payroll and GDP, not vibes.
Read: https://pythonic.ninja/blog/2025-11-15-ev-of-agi-for-western...
it is an architecture problem, too. LLMs simply aren't capable of AGI
Why not?
A lot of people say that, but no one, not a single person has ever pointed out a fundamental limitation that would prevent an LLM from going all the way.
If LLMs have limits, we are yet to find them.
We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI. Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic and physical constraints on continuing this trend indefinitely.
OOD is still unsolved problem, they basically struggle under domain shifts and long tail cases or when you try systematically new combinations of concepts (especially on reasoning heavy tasks). This is now a well documented limitation of LLMs/multimodal LLMs.
Work on COT faithfulness shows that the step by step reasoning they print doesn't match their actual internal computation, they frequently generate plausible but misleading explanations of their own answers (lookup anthropic paper). That means they lack self knowledge about how/why they got a result. I doubt you can get AGI without that.
None of this proves that no LLM based architecture could ever reach AGI. But it directly contradicts the idea that we haven't found any limits. We've already found multiple major limitations of the current LLMs, and there's no evidence that blindly scaling this recipe is enough to cross from very capable assistant to AGI.
A lot of those failings (i.e. COT faithfulness) are straight up human failure modes.
LLMs failing the same way as humans do on the same tasks as humans is a weak sign of "this tech is AGI capable", in my eyes. Because it hints that LLMs are angling to do the same things human mind does, and in similar enough ways to share the failure modes. And human mind is the one architecture we know to support general intelligence.
Anthropic has a more recent paper on introspection in LLMs, by the way. With numerous findings. The main takeaway is: existing LLMs have introspection capabilities - weak, limited and unreliable, but present nonetheless. It's a bit weird, given that we never trained them for that.
https://transformer-circuits.pub/2025/introspection/index.ht...
You can train them to be better at it, if you really wanted to. A few other papers tried, although in different contexts.
This is all nonsense and you are just falling for marketing that you want to be true.
The whole space is largely marketing at this point, intentionally conflating all these philosophical terms because we don't want to face the ugly reality that LLMs are a dead end to "AGI".
Not to mention, it is not on those who don't believe in Santa Clause to prove that Santa Clause doesn't exist. It is on those who believe in Santa Clause to show how AGI can possibly emerge from next token prediction.
I would question if you even use the models much really because I thought this in 2023 but I just can't imagine how anyone who uses the models all the time can possibly think we are on the path to AGI with LLMs in 2025.
It is almost like the idea of a thinking being emerging from text was a dumb idea to start with.
You are falling for the AI effect.
Which is: flesh apes want to feel unique and special! And "intelligence" must be what makes them so unique and special! So they deny "intelligence" in anything that's not a fellow flesh ape!
If an AI can't talk like a human, then it must be the talking that makes the human intelligence special! But if the AI can talk, then talking was never important for intelligence in the first place! Repeat for everything.
I use LLMs a lot, and the improvements in the last few years are vast. OpenAI's entire personality tuning team should be loaded into a rocket and fired off into the sun, but that's a separate issue from raw AI capabilities, which keep improving steadily and with no end in sight.
Breaking down in -30C temperatures is also human failure mode, but doesen't make cars human. They both exhibit the exact same behavior (not moving), but are fundamentally different
The similarities go quite a bit deeper than that.
Both rely on a certain metabolic process to be able to move. Both function in a narrow temperature range, and fail outside it. Both have a homeostatic process that attempts to keep them in that temperature range. Both rely on chemical energy, oxidizing stored hydrocarbons to extract power from them, and both take in O2-rich air, and emit air enriched in CO2 and water vapor.
So, yes, the cars aren't humans. But they sure implement quite a few of the same things as humans do - despite being made out of very different parts.
LLMs of today? They implement abstract thinking the same way cars implement aerobic metabolism. A nonhuman implementation, but one that does a great many of the same things.
Real time learning that doesn't pollute limited context windows.
You can mimic this already. Unreliable and computationally inefficient, but those are not fundamental limitations.
LLMs are bounded by the same bounds computers are. They run on computers so a prime example of a limitation is Rices theorem. Any ‘ai’ that writes code is unable (just like humans) to determine if the output is or is not error free.
This means a multi agent workflow without human that writes code may or may not be error free.
LLMs are also bounded by runtime complexity. Could an llm find the shortest Hamiltionian path between two cities in non polynomial time?
LLMs are bounded by in model context: Could an llm create and use a new language with no context in its model?
Assuming you want to define the goal, "AGI", as something functionally equivalent to part (or all) of the human brain, there are two broad approaches to implement that.
1) Try to build a neuron-level brain simulator - something that is a far distant possibility, not because of compute, but because we don't have a clear enough idea of how the brain is wired, how neurons work, and what level of fidelity is needed to capture all the aspects of neuron dynamics that are functionally relevant rather than just part of a wetware realization
OR
2) Analyze what the brain is doing, to extent possible given our current incomplete knowledge, and/or reduce the definition of "AGI" to a functional level, then design a functional architecture/implementation, rather than neuron level one, to implement it
The compute demands of these two approaches are massively different. It's like the difference between an electronic circuit simulator that works at gate level vs one that works at functional level.
For time being we have no choice other than following the functional approach, since we just don't know enough to build an accurate brain simulator even if that was for some reason to be seen as the preferred approach.
The power efficiency of a brain vs a gigawatt systolic array is certainly dramatic, and it would be great for the planet to close that gap, but it seems we first need to build a working "AGI" or artificial brain (however you want it define the goal) before we optimize it. Research and iteration requires a flexible platform like GPUs. Maybe when we figure it out we can use more of a dataflow brain-like approach to reduce power usage.
OTOH, look at the difference between a single user MOE LLM, and one running in a datacenter simultaneously processing multiple inputs. In the single-user case we conceptualize the MOE as saving FLOPs/power by only having one "expert" active at a time, but in the multi-user case all experts are active all the time handling tokens from different users. The potential of a dataflow approach to save power may be similar, with all parts of the model active at the same time when handling a datacenter load, so a custom hardware realization may not be needed/relevant for power efficiency.
Or
3) Pour enough computation into a sufficiently capable search process and have it find a solution for us
Which is what we're doing now.
The bitter lesson was proven right once again. LLMs prove that you can build incredibly advanced AIs without "understanding" how they work.
You could do an architectural search, and Google previously did that for CNNs with it's NASNet (Network Architectural Search) series of architectures, but the problem is you first need to decide what are the architectural components you want your search process to operate over, so you are baking in a lot of assumptions from the start and massively reducing the search space (because this is necessary to be computationally viable).
A search or evolutionary process would also need an AGI-evaluator to guide the search, and this evaluator would then determine the characteristics of the solution found, so it rather smacks of benchmark gaming rather than the preferred approach of designing for generic capabilities rather than specific evaluations.
I wouldn't say we don't know how LLMs "work" - clearly we know how the transformer itself works, and it was designed intentionally with certain approach in mind - we just don't know all the details of what representations it has learnt from the data. I also wouldn't say LLMs/transformers represent a bitter lesson approach since the architecture is so specific - there is a lot of assumptions baked into it.
Hard problem of consciousness seems way harder to wolve than the easy one which is a purely engineering problem. People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea.
> People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea
I'm not sure what you mean by this.
I think there is a pretty large consensus that our neocortex is a prediction machine (predicting future observations/outcomes from past experience), and the reason WHY it would have evolved to be this is because there is obvious massive survival benefit in successfully predicting how predators and prey will react ahead of time, what will be the outcome of your own actions, etc, etc. Prediction unlocks you from being stuck in the present having to react to things as they happen and lets you plan ahead.
Thinking = Reasoning/Planning is just multi-step prediction.
I don't think consciousness is the big deal most people think it is - it seems to be just the ability to self-observe (which helps to self-predict), but if we somehow built AGI that wasn't conscious, then who cares?
Not why it was created, why the systems in the brain lead to consciousness. Your option B requires understanding not just mechanically how but the fundamental reason for why consciousness appears. If we just understand the mechanics all we can confidently do is work toward a more and more accurate representation of the brain. AGI without consciousness is speculated but hard for me to believe in.
As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input.
I would expect that all intelligent animals are conscious, and any AI we build with a roughly brain-like architecture in terms of connections, looping, and being prediction based would also report itself to be conscious and describe a similar subjective experience. LLMs seem much too simple (just layer-wise pass-thru data flow) to be conscious.
It's possible that some of the neural connections supporting consciousness may have evolved, or been enhanced, due to the evolutionary value of enhanced self-prediction (i.e. this is the reason), but as noted I expect it basically "comes for free" with any complete enough cognitive/sensory architecture.
> As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input.
As far as I know, consciousness is referring to something other than self-referential systems, especially with regards to the hard problem of consciousness.
The [philosophical zombie](https://en.wikipedia.org/wiki/Philosophical_zombie) thought experiment is well-known for imagining something with all the structural characteristics that you mention but without conscious experience as in "what-its-like" to be someone.
It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.
When considering the structural underpinnings of consciousness, it's interesting to note the phenomena of "blindsight", which is essentially a loss of visual consciousness without an actual loss of vision!
Note that anything with mental access to it's own deliberations and sensory inputs will by definition always be able to report "what it's like" to be themselves - what they are experiencing (what's in their mind). If something reports to you their quales of vision or hearing, isn't this exactly what we mean by "what it's like" to be them - how they feel they are experiencing the world?!
> It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.
Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience. We can certainly have an opinion or belief or assume that sufficient structure means consciousness, which is a perfectly reasonable stance to take and one that many would take, but we have to be careful to understand that's not a scientific stance since it isn't testable or falsifiable, which is why it's been called the "hard problem" of consciousness. It's an unfounded belief we choose out of reasons like psychological comfort.
With regards to your latter point, I think you are making some sophisticated distinctions regarding the "map and territory" relation, and it seems you've hit upon the crux of the matter: how can we report "what its like" for us to experience something the other person hasn't experienced, if its not deconstructible to phenomenal states they've already experienced (and therefore constructible for them based off of our report)? The landmark paper here is "What Is It Like to Be a Bat?" by Josh Nagel, and if you're ever curious it's a pretty short read.
With regards to "blindsight" since I'm not familiar with it and curious, how do we distinguish between loss of visual consciousness and loss of information transfer between conscious regions, or loss of memory about conscious experience?
I'm not sure how much, if any, work has been done to study the brains of people with blindsight. I'm also not sure I would differentiate between loss of visual consciousness and loss of information transfer ... my understanding is that it's the loss if information transfer that is causing the loss of consciousness (e.g maybe your visual cortex works fine, so you can see, and you can perform some visual tasks that have been well practiced and/or no longer need general association cortex, but if the connection between visual cortex and association cortex was lost, then perhaps this is where you become unaware of your ability to see, i.e. lose visual consciousness).
I don't think it's a memory issue - one classic test of blindsight is asking the patient to navigate a cluttered corridor full of obstacles, which the patient succeeds in doing despite reporting themselves as blind - so it's a real-time phenomena, not one of memory.
> Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience.
That seems to come down to defining, in a non hand-wavy way, what we mean by "conscious phenomenal experience". If this is referring to personal subjective experience, then why is just asking them to report that subjective experience unsatisfactory ?!
I get that consciousness is considered as some ineffable personal experience, but as a thought experiment, what if the experimenter, defining themselves as "conscious" wanted to probe if some subject's subjective experience differed from their own, then they could at least attempt to verbalize any and all aspects of their own (the experimeter's) subjective experience and ask the subject if they felt the same, and the more (unconstrained) questions they asked without finding any significant difference would make it asymptotically unlikely that there was any difference.
> which is why it's [p-zombie detection] been called the "hard problem" of consciousness
AFAIK the normal definition of the hard problem is basically how and why the brain gives rise to qualia and subjective experience, which really seems like a non-problem...
We have thoughts and emotions, and mental access to these, so it has to feel like something to be alive and experience things. If we introspect on what having, say, vision, is like, or what it is like to have our eyes open vs shut, then (assuming we don't have blindsight!) we are obviously going to experience the difference and be able to report it - it does "feel" like something.
Qualia are an interesting thing to discuss - why do we experience what we do, or experience anything at all for that matter when we see, say a large red circle. Why does red feel "red"? Why and how does music feel different in nature to color, and why does it feel the way it does, etc?
I think these are also really non-problems that disappear as soon as you start to examine them! What are the differences in quales of seeing a small red circle vs a large red circle, or a large blue circle vs a large red one... When you consider differences in quales vs the fact that we experience anything at all (which is proved by our ability to report that we do). Color is perceived as surface attribute with a spatial extent, with colors differentiated by what they remind us of. Blue brings to mind water, sky and other blue things, Red brings to mind fire, roses, and other red things. Perception of color can be proven to be purely associative, not absolute, by Ivo Kohler's chromatic adaptation experiments, having the subject wear colored goggles whose effect "wears off" after a few days with normal subjective perception of color returning.
I'm actually curious here, because maybe our experiences are different. When you look at something red, before any associations or thoughts kick in, before you start thinking "this reminds me of fire" or analyzing it, is there something it's like for that redness to be there? Some quality to it that exists independent of what you can say about it?
For me, I can turn off all the thinking and associations and just... look. And there's something there that the looking is of or like, if that makes sense. It's hard to put into words because it's prior to words, and can possibly be independent of them.
But maybe that's not something universal? I know some people don't have visual imagery or an inner voice, so maybe phenomenal experience varies more than we assume. Does that distinction between the experience itself and your ability to think/talk about it track for you at all?
> And there's something there that the looking is of or like, if that makes sense
I think I know what you mean, but if you consider something really simple like the patch of a single color, even without any color associations (although presumably they are always there subconsciously) then isn't the experience just of "a surface attribute, of given spatial extent". There is something there, that is the same in that spatial region, but different elsewhere.
At least, that's how it seems to me, and isn't that exactly how the quale of a color has to be - that is the essence of it ?!
Yeah that pretty much seems to be it.
> then isn't the experience just of "a surface attribute, of given spatial extent".
I don't know why this seems to be so hard for me to think about and even put into words, but isn't "the experience of the surface attribute of a given spatial extent" something other than the experience of the surface attribute of a given spatial extent itself?
I mean that the words we use to describe something aren't the something itself. Conceivably, you can experience something without ever having words, and having words about a phenomenal visual experience doesn't seem to change the experience much or at all (at least for me).
Maybe another way of phrasing this would be something like: can we talk about red blotches using red blotches themselves, in the same way that we can talk about words using words themselves? And then, supposing that we could talk about red blotches using red blotches (maybe the blotches are in the form of words or structured like knowledge, I dunno), can we talk about red blotches without ever having experienced red blotches? I learned this idea from Mary's Room thought experiment, but I still don't know what to think about it.
Yes - the experience / quale has nothing to do with words.
The point (opinion) I'm trying to make is that something like the quale of vision, that is so hard to describe, basically has to be the way it is, because that is it's fundamental nature.
Consider starting with your eyes closed, and maybe just a white surface in front of you, then you open your eyes. Seeing is not the same as not-seeing, so it has to feel different. If it was a different color then the input to your brain would be different, so that has to feel different too. Vision is a spatial sense - we have a 2-D array of rods and cones in our retina feeding into our brain, so (combined with persistence of vision) we experience the scene in front of us all at once in a spatial manner, completely unlike hearing which is a temporal sense with one thing happening after another... etc, etc.
It seems to me that when you start analyzing it, everything about the quale of vision (or hearing, or touch, or smell) has to be the way it is - it is no mystery - and an artificial brain with similar senses would experience it exactly the same way.
Yep that's a cogent, serious stance, and it sounds a lot like illusionism (famously argued by Daniel Dennett) or functionalism if you ever wanted to check out more about it.
It’s a serious stance, but the really interesting thing to me here is that its not a settled fact. What’s quite surprising and unique about this field is that unlike physics or chemistry where we generally agree on the basics, in consciousness studies you have some quite brilliant minds totally deadlocked on the fundamentals. There is absolutely no consensus on whether the problem is 'solved' or 'impossible,' and its definitely not a matter of people not taking this seriously enough or making some rash judgments or simple errors.
I find this fascinating because this type of situation is pretty rare or unique in modern science. Maybe the fun part is that I can take one stance and you another and here there's no "right answer" that some expert knows and one of us is "clearly" wrong. Nice chatting with you :)
Correct - the vast majority of people vastly underestimate the complexity of the human brain and the emergent properties that develop from this inherent complexity.
exactly, the brain - what a concept! over here you have broca's area, there, wernicke, then Bowman's crest, sector 19, and undiscovered country.
if you put the brain in the shape of a tube you'd have a really long err, well, let's say it's not a good idea to do that. the brain gives me goosepimples, my brain too
>It is ultimately a hardware problem.
I think it's more an algorithm problem. I've been reading how LLMs work and the brain does nothing like matrix multiplication over billions of entities. It seems a very inefficient way to do it in terms of compute use, although efficient in terms of not many lines of code. I think the example of the brain shows one could do far better.
Humans grow over years with plenty of self guided study. It's far more than a hardware problem.
Quantum compute is my guess. Being able to switch entire models at atomic speeds will give the perception of intelligence at least. There is still a lot there that will need to be figured out between now and then.
I remember reading about memristors when I was at University and the hope they could help simulate neurons.
I don't remember hearing much about neuromorphic computing lately though so I guess it hasn't had much progress.
It’s not the level of computing we might hope for, but there has been some progress in developing memristors :)
https://journals.plos.org/plosone/article?id=10.1371/journal...
That's my non-expert belief as well. We are trying to brute force an approximation of one aspect of how neurons work at great cost.
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
Try explaining to someone who's only ever seen dial-up modems that 4k HDR video streaming is a thing.
Dial-up modems can transfer a 4K HDR video file, or any other arbitrary data.
It obviously wouldn't have the bandwidth to do so in a way that would make a real-time stream feasible, but it doesn't involve any leap of logic to conclude that a higher bandwidth link means being able to transfer more data within a given period of time, which would eventually enable use cases that weren't feasible before.
In contrast, you could throw an essentially unlimited amount of hardware at LLMs, and that still wouldn't mean that they would be able to achieve AGI, because there's no clear mechanism for how they would do so.
From modern perspective it's obvious that simply upping the bandwidth allows streaming high-quality videos, but it's not strictly about "more bigger cable". Huge leaps in various technologies were needed for you to watch video in 4k:
- 4k consumer-grade cameras
- SSDs
- video codecs
- hardware-accelerated video encoding
- large-scale internet infrastructure
- OLED displays
What I'm trying to say is that I clearly remember reading an old article about sharing mp3s on P2P networks and the person writing the article was confident that video sharing, let alone video streaming, let alone high-quality video streaming, wouldn't happen in foreseeable future because there were just too many problems with that.
If you went back in time just 10 years and told people about ChatGPT they simply wouldn't believe you. They imagined that an AI that can do things that current LLMs can do must be insanely complex, but once technology made that step, we realized "it's actually not that complicated". Sure, AGI won't surface from simply adding more GPUs into LLMs, just like LLMs didn't emerge from adding more GPUs to "cat vs dog" AI. But if technology took us from "AI can tell apart dog and cat 80% of the time" to "AI is literally wiping out entire industry sectors like translation or creative work while turning people into dopamine addicts en masse" within ten years, then I assume that I'll see AGI within my lifetime.
There's nothing about 4K videos that needs an SSD, an OLED display, or any particular video codec, and "large-scale internet infrastructure" is just a different way of saying "lots of high-bandwidth links". Hardware graphics acceleration was also around long before any form of 4K video, and a video decoding accerator is such an obvious solution that dedicated accelerators were used for early full-motion video before CPUs could reasonably decode them.
Your anecdote regarding P2P file sharing is ridiculous, and you've almost certainly misunderstood what the author was saying (or the author themselves was an idiot). That there wasn't sufficient bandwidth or computing power to stream 4K video at consumer price points during the heyday of mp3 file sharing, didn't mean that no one knew how to do it. It would be as ridiculous as me today saying that 16K stereoscopic streaming video can't happen. Just because it's infeasible today, doesn't mean that it's impossible.
Regarding ChatGPT, setting aside the fact that the transformer model that ChatGPT is built on was under active research 10 years ago, sure, breakthroughs happen. That doesn't mean that you can linearly extrapolate future breakthroughs. That would be like claiming that if we developer faster and more powerful rockets, then we will eventually be able to travel faster than light.
so planes that don't flap their wings can't fly
Exactly why I cringe so hard when AI-bros make arguments equating AI neurons to biological neurons.
There are some tradeoffs in the other direction. Digital neurons can have advantages that biological neurons do not.
For example, if biology had a "choice" I am fairly confident that it would have elected to not have leaky charge carriers or relatively high latency between elements. Roughly 20% of our brain exists simply to slow down and compensate for the other 80%.
I don't know that eliminating these caveats is sufficient to overcome all the downsides, but I also don't think we've tried very hard to build experiments that directly target this kind of thinking. Most of our digital neurons today are of an extremely reductive variety. At a minimum, I think we need recurrence over a time domain. The current paradigm (GPU-bound) is highly allergic to a causal flow of events over time (i.e., branching control flows).
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
Me too. But, I worry this “want” may not be realistic/scalable.
Yesterday, I was trying to get some Bluetooth/BLE working on a Raspberry CM 4. I had dabbled with this 9 months ago. And things were making progress then just fine. Suddenly with a new trixie build and who knows what else has changed, I just could not get my little client to open the HCI socket. In about 10 minutes prompt dueling between GPT and Claude, I was able to learn all about rfkill and get to the bottom of things. I’ve worked with Linux for 20+ years, and somehow had missed learning about rfkill in the mix.
I was happy and saddened. I would not have k own where to turn. SO doesn’t get near the traffic it used to and is so bifurcated and policed I don’t even try anymore. I never know whether to look for a mailing list, a forum, a discord, a channel, the newsgroups have all long died away. There is no solidly written chapter in a canonically accepted manual written by tech writers on all things Bluetooth for the Linux Kernel packaged with raspbian. And to pile on, my attention span driven by a constant diet of engagement, makes it harder to have the patience.
It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
While I generally agree with this, I have mixed feelings. On the one hand, AI could be smart enough to reach the "enlightened master engineer" and can reach super-human levels of...simplification. In some ways, complexity can result from improper layering and abstraction inversion. It takes a holistic view to realize that the lines (i.e. interfaces) between layers were drawn improperly and redesign everything together, achieving an overall simplification.
A good example is the web platform. It's just enormous...to the point that no human can really understand how it all even works. And I say that as someone who worked for a long time on a narrow part of that stack (V8). It being only a little over a million lines of code, it is incredibly intricate and subtle, because it implements a pretty weird language, has lots of optimizations, advanced GC, multiple compilers, etc. And that's just the JS engine. Add in the layout engine, rendering engine, multi-process architecture...it's beyond the comprehension a single mind.
We're not yet at the level that an AI can understand code really deeply yet, but may we will reach the point where an AI understands enough of it and can code competently enough to start over from scratch and build something we can both understand and does the things we actually want it to do.
>our systems cannot withstand shocks
We've seen a disturbing preview of this recently.
It's a natural law that what is not exercised dies away.
When we make our systems too stable and predictable, the ability to operate effectively in the absence of stability also dies away.
I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
Distros do have manuals, they just usually come in the form of user-curated wikis these days. ArchWiki is usually my first stop when I run into a Linux issue, even as a fellow Debian user.
Both https://wiki.archlinux.org/title/Bluetooth and https://wiki.debian.org/BluetoothUser mention rfkill and show you how to troubleshoot.
> It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
This is the real AI risk we should be worried about IMO, at least short term. Information technology has made things vastly more complicated. AI will make it even more incomprehensible. Tax code, engineering, car design, whatever.
It's already happening at my work. I work at big tech and we already have a vast array of overly complicated tools/technical debt no one wants to clean up. There's several initiatives to use AI to prompt an agent, which in turn will find the right tool to use and run the commands.
It's not inconceivable that 10 or 20 years down the road no human will bother trying to understand what's actually going on. Our brains will become weaker and the logic will become vastly more complicated.
Has anyone tried putting the AIs to work cleaning up the technical debt?
Yes, I'm already doing it. But the problem is there's not a lot of incentive from management to do it.
Long term investment in something that can't easily be quantified is a non-starter to management. People will say "thank you for doing that" but those who create new features that drive metrics get promoted.
That's why it's very important to work for an organization run by engineers or former engineers when writing software.
And engineers who actually shipped and maintained successful products over time.
For software developed in-house by non-software organizations, the incentives are all wrong because management cannot properly assess the value of well architected systems.
That's true of smaller orgs -- ones that can't afford to pay for nonsense 5-10 years down the road.
Big tech can afford to pay for all kinds of nonsense.
I see this too. The focus on “believable ROI” makes it very hard to do.
I think LLMs as a replacement for Google, Stack Overflow, etc. is a no brainer. As long as you can get to the source documents when you need them, and train yourself to sniff out hallucinations.
(We already do this constantly in categorizing human generated bullshit information and useful information constantly. So learning to do something similar with LLM output is not necessarily worse, just different.)
What's silly at this point is replacing a human entirely with an LLM. LLMs are still fundamentally unsuited for those tasks, although they may be in the future with some significant break throughs.
Yeah, using LLMs makes me reconsider the complexity of the software I'm producing and I'm relying on. In a sense LLMs can be a test for the complexity and the fast iteration cycles could yield better solutions than the existing ones
The LLMs we have today aren't a fantasy: They're a concrete thing that works.
Just because the people who make them live in a fantasy world, doesn't mean we can't reap the fruits of their labor!
That being said, I suspect a lot of the energy spent on AI training is resulting in unusable slop.
The only reason search might not be as good as an AI is enshitifications. I expect these LLMs will be the same after the open web has withered.
LLMs are quite good at "semantic search", where traditional search engines struggle
No one said search engines need to be "traditional" or only Page Rank or whatever.
But you also don't need to filter the information through an hallucinating UX just so you can use vectorization to match actual results.
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
They're a dead end for whatever their definition of "AGI" is, but still incredibly useful in many areas and not a "dead end" economically.
Well, except for that needing a 40 year bond for a 3 year technology cycle thing.
I figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
"Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
"It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
Name some? Because YeCun's break is the first big name I've seen strike out in a major fashion away from the LLM trajectory.
How can u look at progress in LLMs and think "this is mirage"
Because the depth of concept is so deep, and if you're just beginning then it's going to take a while for that statement to illuminate how obvious it is.
For those who've been sniffing this since early 2010, it's so blindly obvious they've already dropped llms on the floor and moved onto deeper alternative research.
For the rest of us, we're still catching coke bottles from the sky and building places of worship around them
> Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end.
There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
> I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it.
Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes.
So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
Far more common are ideas that don't work on any scale at all.
If you have something that gives a sticky +5% at 250M scale, you might have an actual winner. Almost all new ML ideas fall well short of that.
If someone else comes along and makes the exact claim I just made, I won't believe it either
Did you try any of your shit at any scale at all?
999999 times out of a million you'd be right.
But, I shouldn't have said anything.
Let me guess, you have a truly marvelous demonstration of this proposition which this margin is too narrow to contain.
Well it kind of sucks honestly because im never going to get any sort of recognition, but that's partly by choice and partly because I don't have the right personal connections. I 100% understand why all of you think I'm a significant fool. Thats ok. Like i said, im burying the idea. But its obvious enough that someone else will discover it soon enough.
You literally see no irony here? You are doubling, tripling down on some idea you won't even vaguely describe to people, but you are so utterly convinced that it's right you don't appear to even admit the possibility that your idle thoughts could be wrong, or just silly. Frankly it sounds delusional and you should question yourself more often. If you're so afraid of being right that you won't even write the idea down to have another person look at it for fear it will somehow set the world on fire because of its rightness, then you should seriously consider talking to a therapist. If that comes across as an ad hominem, it isn't, and it isn't meant as an insult or a dismissal of your idea--you've actually offered no idea. Frankly, you should be a little embarrassed as you've done nothing but grouse about how smart you are, and people react negatively to that. Puffing gets us nowhere except drawing attention to yourself and gave us no new ideas or thoughts to discuss, except your ego. I'd recommend you question yourself more and talk to some people you trust about whatever it is you think you discovered.
Tell you what, here's a story from my last job.
I was hired on to help run these experiments with a machine they were trying to perfect. They wanted to do all this stuff with running two machines in parallel, dumping the refuse from one into another machine, all kinds of silly stuff.
I looked at it and in a few minutes I sketched in my notebook a solution to their problem that would achieve the maximum theoretical yield without all these wacky experiments.
But then what would be my job? I was hired to run these experiments not solve their whole problem in 5 minutes. So I didn't tell them. I was getting paid so i didn't care if they were wasting time. I'm a smiley little trans woman anyway, they wouldn't have listened. I did work out the math though on my own.
A few months later, a team of phds at a university we were working with hit on a rudimentary form of the same idea, so at that point I revealed to the boss, oh, look I've worked out the math for that already. And I wrote up the code, and we hit the maximum theoretical yield and it ran 100x faster computationally, and i got a pat on the back. But the reason they listened was because a man with a degree said it.
Why would I bother giving them ideas if they won't listen anyway? If i had given them the answer months earlier, I would not have gotten any kind of bonus even in the unlikely event that I had been taken seriously.
So now of course youre all telling me im insane, and thats fine because honestly I have no incentive to give you answers ahead of time. Go suck eggs. Figure shit out on your own.
> If i had given them the answer months earlier, I would not have gotten any kind of bonus even in the unlikely event that I had been taken seriously.
You guaranteed that you wouldn't get recognition by keeping the idea to yourself. Your victimization was by your own design, and it stayed where it started: in your head. And ironically you were the one being passive agressive and showing bias towards your coworkers.
Oh honey. I wasn't born with this attitude you know- it was beaten into me.
Nothing is punished so swiftly and severely as outshining your boss or making him look stupid.
Ugh why do you care so much? Leave me alone. The big insane statement i made was "i have some idea of what the next bit of advancement will look like". Transformers are a special case of a much more beautiful general principle ok? Go away
cringe
Reject subtlety
Embrace cringe
[dead]
[flagged]
If you're so confident that your discovery will be found by someone else soon why not just share it right here and get the credit you seem to want? Otherwise why even bring it up in the first place?
No reason to risk making the humanity killing doomsday machine appear even a minute sooner, and besides, you wouldn't listen.
I said something because im a fool who cant keep her mouth shut. It was reasons of emotional catharsis having no rational basis.
You're being downvoted for displaying the kind of overconfidence that people consider shameful.
Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
I know how it looks. I shouldn't have said anything- it was pointless.
At any rate someone else will figure out what i know soon or already has.
Nostradamus 2.0
Nah im just the kind of autistic that is good at solving specific kinds of problems that can be worked out through visualizing data structures
[dead]
I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
Author here. Indeed - it would be just as fantastical to deny there has been no value from deep learning, transformers, etc.
Yesterday I heard Cory Doctorow talk about a bunch of pro bono lawyers using LLMs to mine paperwork and help exonerate innocent people. Also a big win.
There's good stuff - engineering - that can be done with the underlying tech without the hyperscaling.
Not only whispr, so much of the computer vision area is not as in vogue. I suspect because the truly monumental solutions unlocked are not that accessible to the average person; i.e. industrial manufacturing and robotics at scale.
I think that LLM hype is hiding a lot of very real and impactful progress in real world/robot intelligence.
An essay writing machine is cool. A machine that can competently control any robot arm, and make it immediately useful is a world-changing prospect.
Moving and manipulating objects without explicit human coded instructions will absolutely revolutionize so much of our world.
That's because industrial manufacturing and robotics are failing to bring down costs and make people's lives more affordable.
That's really the only value those technologies provide, so if people aren't seeing costs come down there really is zero value coming from those technologies.
I switched to Parakeet the other day.
It's better than Whisper, and faster, while running on CPU on my ten year old ThinkPad.
I had Claude make me Python bindings for it and add it to my voice typing app.
We live in the future.
I think a lot of AI wins are going to end up local and free much like whisper.
Maybe it could be a little bit more accurate, it would be nice if it ran a little faster, but ultimately it's 95% complete software that can be free forever.
My guess is very many AI tasks are going to end up this way. In 5-10 years we're all going to be walking around with laptops with 100k cores and 1TB of RAM and an LLM that we talk to and it does stuff for us more or less exactly like Star Trek.
What do you think about the shape of AI progress?
Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
I will go meta into what you posted here: That people are classifying themselves as "AI skeptics". Many people are treating this in terms of tribal conflict and identity politics. On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits. If we do talk about it as a debate, we can do it when with open minds, and intellectual honesty.
I think much of this may be a reaction to the hype promoted by tech CEOs and media outlets. People are seeing through their lies and exaggerations, and taking positions like "AI/LLMs have no values or uses", then using every argument they hear as a reason why it is bad in a broad sense. For example: Energy and water concerns. That's my best guess about the concern you're braced against.
> I will go meta into what you posted here: That people are classifying themselves as "AI skeptics"
The comment you're replying to is calling other people AI skeptics.
Your advice has some fine parts to it (and simonw's comment is innocuous in its use of the term), but if we're really going meta, you seem to be engaging in the tribal conflict you're decrying by lecturing an imaginary person rather than the actual context of what you're responding to.
To me, "Tip for AI skeptics" reads as shorthand for "Tip for those of you who classify as AI skeptics".
That is why the meta commentary about identity politics made complete sense to me. It's simply observing that this discussion (like so many others) tends to go this way, and suggests a better alternative - without a straw man.
I read it more as a claim that people who advocate against AI are picking arguments as a means to an end rather than because they actually believe or care about what they're saying.
Expecting a purely technical discussion is unrealistic because many people have significant vested interests. This includes not only those with financial stakes in AI stocks but also a large number of professionals in roles that could be transformed or replaced by this technology. For these groups, the discussion is inherently political, not just technical.
I don't really mind if people advocate for their value judgements, but the total disregard for good faith arguments and facts is really out of control. The number of people who care at all about finding the best position through debate and are willing to adjust their position is really shockingly small across almost every issue.
Totally agree. It seems like a symptom of a larger issue: people are becoming increasingly selfish and entrenched in their own bubbles. It’s hard to see a path back to sanity from here.
Well, I share your pain .. but was it ever really better in reality?
Unfortunately it is not like human society in history had truth as the highest virtue.
Human societies? No.
Subcultures? Some are at least trying to (i.e. rationalists), though imperfectly and with side-effects.
This depends on the particular group of rationalists. An unfortunately outsized and vocal group with strong overlap in the tech community has gone to notions of quasi mathematical reasoning distorting things like EV ("expected value"). Many have stretched "reason" way past the breaking point to articles of faith but with a far more pernicious affect than traditional points of religious dogma that are at least more easily identifiable as "faith" due to their religious trappings.
Edit: See Roko's Basilisk as an example. wherein something like variation on Christian hell is independently reinvented for those not donating enough to bring about the coming superhuman AGI, who will therefore punish you- or the closest simulation it can spin up in VR if you're long gone- for all eternity. The infinite negative EV far outweighing any positive EV of doing more than subsist in poverty. Even managed to work in that it could be a reluctant, but otherwise benevolent super AI such that, while benevolent, it wanted to exist, and to maximize its chances it bound itself to a promise in the future to do these things as an incentive for people to get it to exist.
yeah maybe around the time of Archimedes it was closer to the top, but societies in which people are willing to die for abstract ideas tend to be one... where the value of life isn't quite as high as it is nowadays (ie no matter how much my inner nerd has a love and fascination for that time period, no way i'm pressing the button on any one-way time machines...).
Archimedes, who according to legend was killed in the middle of searching for truth?
>yeah maybe around the time of Archimedes
I mean, Archimedes stands out because he searched for the truth and documented it. I'm sure most people on the planet at that time would have burned you for being a witch, or whatever fabled creature was in vogue at the time.
Only among the people who are yelling, perhaps? I find the majority of people I talk with have open minds and acknowledge the opinions of others without accepting them as fact.
> a large number of professionals in roles that could be transformed or replaced by this technology.
Right, "It is difficult get a man to understand something when his salary depends on his not understanding it."
I see this sort of irrationality around AI at my workplace, with the owners constantly droning on about "we must use AI everywhere." They are completely and irrationally paranoid that the business will fail or get outpaced by a competitor if we are not "using AI." Keep in mind this is a small 300 employee, non-tech company with no real local competitors.
Asking for clarification or what they mean by "use AI" they have no answers, just "other companies are going to use AI, and we need to use AI or we will fall behind."
There's no strategy or technical merit here, no pre-defined use case people have in mind. Purely driven by hype. We do in fact use AI. I do, the office workers use it daily, but the reality is it has had no outward/visible effect on profitability, so it doesn't show up on the P&L at the end of the quarter except as an expense, and so the hype and mandate continues. The only thing that matters is appearing to "use AI" until the magic box makes the line go up.
I've heard the same breathless parroting of the marketing hype at large O(thousands ppl) cloud tech companies. A quote from leadership:
> This is existential. If we aren't early adopters of AI tools we will be left behind and will never catch up.
This company is dominant in the space they operate in. The magnitude of the delusion is profound. Ironically, this crap is actually distracting and affects quality, so it could affect competitiveness--just not how they hope.
I've seen the same trend. AI neeeds to be everywhere, preferably yesterday, but apart from hooking everything up to an LLM withot regards for the consequences nobody seems to know what the AI is supposed to do.
| Drop the politics
Politics is the set of activities that are associated with making decisions in groups, or other forms of power relations among individuals, such as the distribution of status or resources.
Most municipalities literally do not have enough spare power to service this 1.4 trillion dollar capital rollout as planned on paper. Even if they did, the concurrent inflation of energy costs is about as political as a topic can get.
Economic uncertainty (firings, wage depression) brought on by the promises of AI is about as political as it gets. There's no 'pure world' of 'engineering only' concerns when the primary goals of many of these billionaires is leverage this hype, real and imagined, into reshaping the global economy in their preferred form.
The only people that get to be 'apolitical' are those that have already benefitted the most from the status quo. It's a privilege.
Hear hear, It's funny having seen the same issue pop up in video game forums/communities. People complaining about politics in their video games after decades of completely straight faced US military propaganda from games like Call of Duty but because they agree with it it wasn't politics. To so many people politics begins where they start to disagree.
There are politics and there are Politics, and I don't think the two of you are using the same definition. 'Making decisions in groups' does not require 'oversimplifying issues for the sake of tribal cohesion or loyalty'. It is a distressingly common occurrence that complex problems are oversimplified because political effectiveness requires appealing to a broader audience.
We'd all be better off if more people withheld judgement while actually engaging with the nuances of a political topic instead of pushing for their team. The capacity to do that may be a privilege but it's a privilege worth earning and celebrating.
My definition is the definition. You cannot nuance wash the material conditions that are increasing tribal polarization. Rising inequality and uncertainty create fear and discontent, people that offer easy targets for that resentment will have more sway.
The rise of populist polemic as the most effective means for driving public behavior is also downstream from 'neutral technical solutions' designed to 'maximize engagement (anger) to maximize profit'. This is not actually a morally neutral choice and we're all dealing with the consequence. Adding AI is fuel for the fire.
The negatively coded, tribal/political speech can be referred to as 'Polemic' which stems from 'warlike' expression.
And polemic is an entirely legitimate form of political action.
Would you rather starve or never lie about those that would starve you?
I would rather not trust the first person who claims <outgroup> wants to starve me. Polemnics may be legitimate - they may not be, I haven't thought about it deeply - but they are undoubtedly worth dropping from my own information diet.
IMO this is one of those areas where we (English speakers, likely most other languages as well) collectively suffer from ambiguous words.
It might be more accurate to say "drop the partisanship" or "drop the in/out-group generalizations", etc.
I mean, it is intellectually honest to point out that the AI debate at the point is much more a religious or political than strictly technical really. Especially the way tech CEOs hype this as the end of everything.
> IMO the move is drop the politics, and discuss things on their technical merits.
I'd love this but it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
It's not unlike when religious people condemn a book they refuse to read. The merits of the book don't matter, it's symbolic opposition to something broader.
Okay, but a lot of people are calling environmental and content theft arguments "political" in an attempt to make it sound frivolous.
It's fine if you think every non-technical criticism against AI is overblown. I use LLMs, but it's perfectly fine to start from a place of whether it's ethical, or even a net good, to use these in the first place.
People saying "ignoring all of those arguments, let's just look at the tech" are, generously, either naive or shilling. Why would we only revisit these very important topics, which are the heart of how the tech would alter our society, after it's been fully embraced?
Well they're separate issues. Someone could plausibly take the position that air travel should be banned for environmental reasons, but that has no relevance to the utility of air travel. If a group of people were loudly proclaiming that planes were not only bad but useless, anyone who routinely uses planes would obviously find them non-credible.
They're not separate at all, especially if the question is "How hard should we push people to use this."
To be clear, there are a lot of people who routinely used airplanes who don't post-covid, but insisted that they had to. Yeah, I think it's pretty wasteful to fly across the country for a 30 minute meeting. Most don't fly at all. I don't know what mass-psychosis white collar industries were under to think that was necessary.
They're 100% separate. "Planes aren't useful" and "planes don't work" are completely different sentiments than "it's pretty wasteful to fly across the country for a 30 minute meeting" and "we shouldn't push people hard to use air travel".
I know for a fact that planes work, because I've been on a plane and observed it lifting me high off the ground and rapidly transporting me to a distant location. The fact that planes typically emit CO2 doesn't make their existence and utility some kind of mass hallucination.
This distinction may sound a bit silly, because I assume we all agree that planes literally work. But the point I'm making is as it applies to AI. Like many people, I know from experience that AI isn't vaporware and is extremely useful for many purposes. I'm sure many others haven't had the same experience for various reasons, and factually report their observations in good faith — but that's different from pushing a narrative which one wishes to be true, regardless of how valid the reasons for that wish may be.
> I know for a fact that planes work, because I've been on a plane and observed it lifting me high off the ground and rapidly transporting me to a distant location. The fact that planes typically emit CO2 doesn't make their existence and utility some kind of mass hallucination.
You're arguing with an imaginary person. Read what I wrote. I didn't call AI vaporware, I said we shouldn't consider its integration into society purely on technical merits, ignoring the cost, which I think could be big if OpenAI's very public plans are made reality. You're making a strawman.
The mass hallucination is not that plane's are useful, it's that a plane is the only reasonable solution to human communication.
Honestly, you're just further illustrating the complete erosion of nuance that comes when you paint people with concerns about AI as frivolous.
There's no straw man. If what you're saying is you agree with me, then you're the one imagining an argument where none exists.
The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
Driving a massive truck in the city is stupid too and most short flights should be replaced with high speed rail. And AI wastes a monumental amount of resources.
> The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
I keep seeing arguments like this. They sound like a bit like a form of nihilsm. Do you really think we shouldn't worry about risks to the environment simply because we're all hypocrites on that front in one way or another? I get the frustration and have been guilty of using this type of argument myself in the past, but refusing to discuss a problem because the people raising the concern are imperfect human beings doesn't seem like a tenable position.
debating the environmental and public health effects of AI negates the possibility of debating those same things with respect to cars/trucks ?
Charitably, I think you can read into that a not-unreasonable (if unproven) assertion that there are many lower hanging fruits on the tree that would do endlessly more good for the cause to pick than data centers, and AI at least has the arguable potential upside of alleviating some of those specific burdens-- better health care and less environmental pollution through various improved forms of automation. Or at least when addressing the stub claim of the sort in the GP comment, you should assume these fairly straightforward subclaims pre-emptively and respond to this stronger form of the argument. It saves time, at least if you're going to seek out a discussion it does. Plenty of counter claims to them, but it gets the conversational ball rolling in a productive direction and if the response in turn is less constructive then you also know not to bother any more.
I think that form of argument is called "whataboutism". Whether flights waste energy or are environmentally unfriendly is really a separate issue. Both things can be bad.
I wouldn't ignore those arguments but most of the time, they're so poorly formed (eg. using data without logic), they aren't really worth listening to. If you believe AI provides no value, then any environmental cost is too high for you but you can't convey that by trying to dramatize how high it is. That's dishonest and I think people rightly turn off it.
Right now Open"AI", Oracle and everyone else are burning billions of dollars to buy and run these llms, they raise the price of energy around them, they provide negative economic benefit. It's dishonest of you to pretend that isn't the case.
We’ve all used the tools, and they’re… fine. They probably will contribute modestly to overall productivity in certain fields, but they certainly aren’t as transformative or magical as the current hype suggests. I’m not sure why you insist that we continue to fawn over these things.
> I’m not sure why you insist that we continue to fawn over these things.
I'm not sure why all the replies under this comment are full of projections of extreme opinions I do not hold and never said I did.
If you want to have a conversation with someone who thinks like this you can probably find one in this very thread, so why do you respond to me?
No, we've used it, you are creating a strawman argument assuming "AI skeptics" are illiterate and/or incapable of understanding. You ironically are the one refusing to accept the possibility that you are wrong.
> No, we've used it, you are creating a strawman argument
There exists a class of "ai-skeptic" who proudly proclaim they have never and will never use AI. Examples are not hard to find, though I see them more on reddit/instagram/bluesky than I do on HN.
If that does not describe you then my comment is not about you.
The strawman side of it is that one can pick on some extremist and "beat them up" publicly as if that settled something but it doesn't.
Maybe you've used it-- but a very large number of the AI skeptic comments I see that actually cite particular experiences, even comments in the pages of HN, amount to things like, "ChatGPT hallucinated when I asked about the local price of product X and if it was in stock anywhere around. How can anyone take LLM and AI seriously?"
Or worse, things like "Real science and real engineering doesn't rely on tools that behave randomly.".
> it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
Why? Would you say the same if the topic was about recreational drugs? Or, to bring it closer to home, if the topic was about social media?
I think you're being disingenuous by making the analogy to religious people refusing to read a certain book. A book is a foundational source of information. OTOH, one can be informed about GenAI without having used GenAI; you can study the math behind the model, the transformer architecture, etc---the foundational sources of information on this topic. If our goal is to "drop the politics, and discuss things on their technical merits" well I don't see how it can get more purely technical than that.
The frustrating thing is when you're debating people who firmly believe that generative AI "has no utility"... but also refuse to ever try it themselves.
(Which they might even justify because they've read the transformer paper or whatever. That doesn't help inform you if these things actually have practical applications!)
It seems like there is a very strong correlation between identity politics and "AI skepticism."
I have no idea why.
I don't think that the correlation is 1, but it seems weirdly high.
Yep. Same for the other direction: there is a very strong correlation between identity politics and praising AI on Twitter.
Then there's us who are mildly disappointed on the agents and how they don't live their promise, and the tech CEOs destroying the economy and our savings. Still using the agents for things that work better, but being burned out for spending days of our time fixing the issues the they created to our code.
IMO this is a Rorschach test for the politically obsessed because I can't stand politics and have no clue what you are talking about.
You have just trained your brain to be so obsessed with politics you can't but help to see it everywhere.
The adoption and use of technology frequently (even typically) has a political axis, it's kind of just this weird world of consumer tech/personal computers that's nominally "apolitical" because it's instead aligned to the axis of taste/self-identity so it'll generate more economic activity.
AI hating is part of the omnicause because it overlaps with art ho socialism, degrowth environmentalism, and general tech skepticism/ludditism.
[dead]
> On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits.
Zero obligation to satisfy HN audience; tiny proportion of the populace. But for giggles...
Technical merits: there are none. Look at Karpathy's GPT on Github. Just some boring old statistics. These technologies are built on top of mathematical principles in textbooks printed 70-80 years ago.
The sharding and distribution of work across numerous machines is also a well trodden technical field.
There is no net new discovery.
This is 100% a political ploy on the part of tech CEOs who take advantage of the innumerate/non-technical political class that holds power. That class is bought into the idea that massive leverage over resource markets is a win for them, and they won't be alive to pay the price of the environmental destruction.
It's not "energy and water" concerns, it's survival of the species concerns obfuscated by socio-political obligations to keep calm carry on and debate endlessly, as vain circumlocution is the hallmark of the elders whose education was modeled on people being VHS cassettes of spoken tradition, industrial and political roles.
IMO there is little technical merit to most software. Maps, communication. That's all that's really needed. ZIRP era insanity juiced the field and created a bunch of self-aggrandizing coder bros whose technical achievements are copy-paste old ideas into new syntax and semantics, to obfuscate their origins, to get funded, sell books, book speaking engagements. There is no removing any of this from politics as political machinations gave rise to the dumbest era of human engineering effort ever.
The only AI that has merit is robotics. Taking manual labor of people that are otherwise exploited by bougie first worlders in their office jobs. People who have, again with the help of politicians, externalized their biologies real needs on the bodies of poorer illiterates they don't have to see as the first-world successfully subjugated them and moved operations out of our own backyard.
Source: was in the room 30 years ago, providing feedback to leadership how to wind down local manufacturing and move it all over to China. Powerful political forces did not like the idea of Americans having the skills and knowledge to build computers. It ran afoul of their goals to subjugate and manipulate through financial engineering.
Americans have been intentionally screwed out of learning hands on skills with which they would have political leverage over the status quo.
There is no removing politics from this. The situation we are in now was 100% crafted by politics.
Hey Simon, author here (and reader of your blog!).
I used to share your view, but what changed my mind was reading Hao's book. I don't have it to hand, but if my memory serves, she writes about a community in Chile opposing Google building a data centre in their city. The city already suffers from drought, and the data centre, acccording to Google's own assessment, would abstract ~169 litres of water a second from local supplies - about the same as the entire city's consumption.
If I also remember correctly, Hao also reported on another town where salt water was being added to municipal drinking water because the drought, exacerbated by local data centres, was so severe.
It is indeed hard to imagine these quantities of water but for me, anything on the order of a town or city's consumption is a lot. Coupled with droughts, it's a problem, in my view.
I really recommend the book.
The fact that certain specific data centres are being proposed or built in areas with water issues may be bad, but it does not imply that all AI data centres are water guzzling drain holes that are killing Earth, which is the point you were (semi-implicitly) making in the article.
What is it that you imagine happens to the water after it goes through the data center?
Clearly it vanishes without a trace and simply leaves the water cycle.
Just because it doesn’t leave the cycle doesn’t mean it’s not an issue. Where it comes back down matters and as climate change makes wet places wetter and dry places drier, that means it’s less distributed
That said, the water issue is overblown. Most of the water calculation comes from power generation (which uses a ton) and is non-potable water.
The potable water consumed is not zero, but it’s like 15% or something
The big issue is power and the fact that most of it comes from fossil fuels
>The big issue is power and the fact that most of it comes from fossil fuels
This has almost zero to do with the data centers themselves and instead the politicians we vote in.
Simply put we need more 'clean' power generation any way you go about it. Economic growth and production is based on ones ability to produce power. We've been coasting on increasing efficiency for a long time, but we've been in need of larger sources of power and distribution for decades, data centers or not.
With that said DC's shouldn't be built in west texas where it's dry. East of the Mississippi gets enough rain that you build a reservoir and you'll have more than enough water to feed the DC for decades.
Is it more of a pricing problem? If data centers paid the same for water as I do they’d be far more efficient with it. Likewise for power. Giving sweet deals to these things based on promises of jobs and tax revenue seems a bad idea at this point.
The way they measure water consumption is genuinely unbelievably misleading at best. For example measuring the water evaporated from a dams basin if any hydroelectric power is used.
Counting water is genuinely just asinine double counting ridiculousness that makes down stream things look completely insane. Like making a pound of beef look like it consumes 10,000L of water.
In reality of course running your shower for 10 to 15 hours is no where near somehow equivalent to eating beef lasagna for dinner and we would actually have a crisis if people started applying any optimization pressure on these useless metrics.
My favourite example is the complaints about “e-waste” because AirPod batteries are not replaceable. Never mind that the batteries make up about half of the weight of the things ro begin with, the entire annual production of AirPods and their cases would barely fill a single shipping container.
No, not a container ship, or anything substantial like that, just one container (1).
Yet, it makes people lose their rational minds and start foaming at the mouth about Apple’s “wasteful practices”.
Do you have a citation for that factoid?
Apologies, looks like I mis-remembered the numbers from the original debate, the actual number is about 110x 40-foot containers. It can be calculated easily enough from the # shipped, the volume of a charging case, and the volume of a shipping container.
It doesn’t alter the conclusion that much: in the grand scheme of human industrial activity, this is nothing.
I.e.: if AirPods were made by an independent company, that firm would be in the top 50 of the largest corporations in the world!
For example, a comparable sized company would be Coca Cola, which goes through 300,000 tons of aluminium annually for their cans, not to mention oil used for plastic bottles!
Sure, I’m with you on your larger point.
Drinking water does not magically appears in the water cycle the next day.
[0] - "And what we found is is that up to 43% of data centers, and this is our largest data centers, are located in areas of high or extremely high water stress. And that's really shocking because data centers require huge amount of drinking water to be able to cool their servers."
[0]- Business Insider | Exposing The Dark Side of America's AI Data Center Explosion - https://youtu.be/t-8TDOFqkQA?t=1201
Areas of "high water stress" sound very sunny, which presumably is great news for renewable power generation.
"I took all your money and gambled it away, but don't worry because it wasn't destroyed, it's still circulating in the economy and will come back someday."
A reduction in what can be used from the summer snowmelt is a problem, regardless of whether equivalent atoms are redeposited in the winter snowfall.
I’m sure there was some planning commission process involved with the development of these sites. I’m curious if anyone has bothered to look at those meeting minutes to see if there are some material misrepresentation of the water and power needs. I’m going to guess that answer is no.
An update: there seems to be a critical error in the reporting of metrics in this book for the water issue. https://andymasley.substack.com/p/empire-of-ai-is-wildly-mis....
I do think the whole water issue as pointed out in the book is completely discredited with this call out.
The author Karen Hao acknowledges this:
https://x.com/_KarenHao/status/1990412118584144113
Thanks for the recommendation, added to my reading list.
The mistake you are making is letting the author choose your points of comparison, without having a high-level picture of where water usage goes. Comparing water usage to a city is misleading because cities don't use much water; large-scale water use is entirely dominated by agriculture.
I'm conflicted. Zooming out, the problem isn't with AI specifically but economic development in general. Everything has a side effect.
For decades we've been told we shouldn't develop urban centers because of how it development affects local communities, but really it just benefited another class of elites (anonymous foreign investors), and now housing prices are impoverishing younger generations and driving homelessness.
Obviously that's not a perfect comparison to AI, which isn't as necessary, but I think the anti-growth argument isn't a good one. Democracies need to keep growing or authoritarian states will take over who don't care so much about human rights. (Or, authoritarian governments will take over democracies.)
There needs to be a political movement that's both pro-growth and pro-humanity, that is capable of making hard or disruptive decisions that actually benefits the poor. Maybe that's a fantasy, but again, I think we should find ways to grow sustainably.
Not just the poor, how about the bottom 99%? This is what's so frustrating to me about the culture wars and identity politics. Regardless of ones views on the hot button cultural issue du jour, at best they are a distraction, and at worst actively exploited by moneyed interests as a political smokescreen to prevent changes that would be obvious wins for super majorities of the population if analyzed and viewed through a more sober and objective lens of the net effects.
IIRC Google chose to pull out altogether to punish the locals for standing up to them— even though they happily built air-cooled data centers elsewhere.
I mean yes, almost all corporations that have a choice do this. Walmart is one of the better known ones that will put a store right on the edge of a municipality that doesn't want one and cause all kinds of issues for the city at hand.
None of which have to do with AI or AGI.
Nestle is and has been 10000x worse for global water security than all other companies and countries combined because nobody in the value chain cares about someone else’s aquifer.
It’s a social-economic problem of externalities being ignored , which transcends any narrow technological use case.
What you describe has been true for all exported manufacturing forever.
I think the point is: where does this end? Do we continue to build orders-of-magnitude bigger models guzzling orders-of-magnitude more water and other resources, in pursuit of the elusive AGI?
At some we need to end this AGI" rat race and focus on realizing practical benefits from the models we currently have.
Well then vote with your wallet and convince everyone you know to stop the train.
As long as there’s a market, machines are going to continue toto displace labor. That’s not going to stop
Is the argument being made here, "Everybody's doing it"? God help us.
My interpretation was "If an industry that actively works to harm the global health of humanity through their addictive and unhealthy food products is using way more water and we're OK with it, maybe we should give a pass to the industry using a fraction of that water to improve human productivity."
Ton of nuance in my characterizations of both industries, of course, but to a first approximation they are accurate.
> and we're OK with it
Are we? Is anybody? Criticism doesn't need to be directed towards one thing at a time.
Well, the food industry continues churning billions in profits at the expense of our health, so statistically speaking, looks like "we" are OK with!
Totally agreed that criticism should be directed where it's due. But what this thread is saying is that criticism of GenAI is misdirected. I haven't seen nearly as much consternation over e.g. the food industry as I'm seeing over AI -- an industry that increasingly looks like its utility exceeds its costs.
(If the last part sounds hypothetical, in past comments I've linked a number of reports, including government-affiliated sources, finding noticeable benefits from GenAI adoption.)
Have you considered that there is a difference between:
- food, a thing that literally every human needs every 24 hours (really 6-12) to continue to live
- GenAI, a new product with dubious value that contributes significantly to the systemic enshittification of the US and global economy?
FYI whataboutism is a well known (and honestly quite lazy) fallacy and propaganda strategy: https://en.wikipedia.org/wiki/Whataboutism
I should have been more precise with my terms, but there is a difference between "food" and the "food industry" indicated by the likes of Nestle. Yes, everybody needs food. No, nobody needs the ultraprocessed junk Nestle produces.
I didn't see the OP's point as whataboutism, but rather putting things into perspective. We are debating the water usage of a powerful new technology that a large fraction of the world is finding useful [1], which is a fraction of what other, much more frivolous (golf courses!) or even actively harmful (Nestle!) industries use.
[1] https://news.ycombinator.com/item?id=45794907 -- Recent thread with very rough numbers which could well be wrong, but the productivity impact is becoming detectable at a national level.
No it’s a red herring (https://en.wikipedia.org/wiki/Red_herring)
It’s a fundamentally flawed argument
What a weird stance. You agree to mislead or distract from a relevant or important question?
Attempt generosity. Can you think of another way to interpret the comment above yours? Is it more likely they are calling their own argument a red herring, or the one they are responding to?
If something looks like a "weird stance", consider trying harder to understand it. It's better for everyone else in the conversation.
I was supporting the GPs premise directly
Just because there are worse abuses elsewhere doesn't mean datacenters should get a pass.
Golf and datacenters should have to pay for their externalities. And if that means both are uneconomical in arid parts of the country then that's better than bankrupting the public and the environment.
From https://www.newyorker.com/magazine/2025/11/03/inside-the-dat...
> I asked the farmer if he had noticed any environmental effects from living next to the data centers. The impact on the water supply, he told me, was negligible. "Honestly, we probably use more water than they do," he said. (Training a state-of-the-art A.I. requires less water than is used on a square mile of farmland in a year.) Power is a different story: the farmer said that the local utility was set to hike rates for the third time in three years, with the most recent proposed hike being in the double digits.
The water issue really is a distraction which harms the credibility of people who lean on it. There are plenty of credible reasons to criticize data enters, use those instead!
The other reason water usage is a bad thing to focus on is that datacenters don't inherently have to use water. It's not like servers have a spigot where you pour water in and it gets consumed.
Water is used in modern datacenters for evaporative cooling, and the reason it's used is to save energy -- it's typically around 10% more energy efficient overall than normal air conditioning. These datacenters often have a PUE of under 1.1, meaning they're over 90% efficient at using power for compute, and evaporative cooling is one of the reasons they're able to achieve such high efficiency.
If governments wanted to, they could mandate that datacenters use air conditioning instead of evaporative cooling, and water usage would drop to near zero (just enough for the restrooms, watering the plants, etc). But nobody would ever seriously suggest doing this because it would be using more of a valuable resource (electricity / CO2 emissions) to save a small amount of a cheap and relatively plentiful resource (water).
> The water issue really is a distraction which harms the credibility of people who lean on it
Is that really the case? - "Data Centers and Water Consumption" - https://www.eesi.org/articles/view/data-centers-and-water-co...
"...Large data centers can consume up to 5 million gallons per day, equivalent to the water use of a town populated by 10,000 to 50,000 people..."
"I Was Wrong About Data Center Water Consumption" - https://www.construction-physics.com/p/i-was-wrong-about-dat...
"...So to wrap up, I misread the Berkeley Report and significantly underestimated US data center water consumption. If you simply take the Berkeley estimates directly, you get around 628 million gallons of water consumption per day for data centers, much higher than the 66-67 million gallons per day I originally stated..."
Also from that article:
> U.S. data centers consume 449 million gallons of water per day and 163.7 billion gallons annually (as of 2021).
Sounds bad! Now let's compare that to agriculture.
USGS 2015 report: https://pubs.usgs.gov/fs/2018/3035/fs20183035.pdf has irrigation at 118 billion gallons per day - that's 43,070 billion gallons per year.
163.7 billion / 43,070 billion * 100 = 0.38 - less than half a percentage point.
It's very easy to present water numbers in a way that looks bad until you start comparing them thoughtfully.
I think comparing data center water usage to domestic water usage by people living in towns is actually quite misleading. UPDATE: I may be wrong about this, see following comment: https://news.ycombinator.com/item?id=45926469#45927945
Agriculture feeds people, Simon.
It's fair to be critical of how the ag industry uses that water, but a significant fraction of that activity is effectively essential.
If you're going to minimize people's concern like this, at least compare it to discretionary uses we could ~live without.
The data's about 20 years old, but for example https://www.usga.org/content/dam/usga/pdf/Water%20Resource%2... suggests we were using over 2b gallons a day to water golf courses.
The vast majority of water in agriculture goes to satisfy our taste buds, not nourish our bodies. Feed crops like alalfa consume huge amounts of water in the desert southwest but the desert climate makes it a great place to grow and people have an insatiable demand for cattle products.
We could feed the world with far less water consumption if we opted not to eat meat. Instead, we let people make purchasing decisions for themselves. I'm not sure why we should take a different approach when making decisions about compute.
> We could feed the world with far less water consumption if we opted not to eat meat.
If you look at the data for animals, that’s not really true. See [1] especially page 22 but the short of it is that the vast majority of water used for animals is “green water” used for animal feed - that’s rainwater that isn’t captured but goes into the soil. Most of the plants used for animal feed don’t use irrigation agriculture so we’d be saving very little on water consumption if we cut out all animal products [2]. Our water consumption would even get a lot worse because we’d have to replace that protein with tons of irrigated farmland and we’d lose the productivity of essentially all the pastureland that is too marginal to grow anything on (50% of US farmland, 66% globally).
Animal husbandry has been such a successful strategy on a planetary scale because it’s an efficient use of marginal resources no matter how wealthy or industrialized you are. Replacing all those calories with plants that people want to actually eat is going to take more resources, not less, especially when you’re talking about turning pastureland into productive agricultural land.
[1] https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1...
[2] a lot of feed is also distiller’s grains used for ethanol first before feeding them to animals, so we’d wouldn’t even cut out most of that
That paper makes the opposite argument than you thought it made. Even from a freshwater perspective it’s more water-efficient to get calories/protein/fat directly from crops than from animal products.
Since you like their work, the authors of your paper answered that question more generally here https://link.springer.com/article/10.1007/s10021-011-9517-8 where they conclude "The water footprint of any animal product is larger than the water footprint of crop products with equivalent nutritional value".
You make some often debunked claims, like we'd have to plant more crops to feed humans directly if we stopped eating meat.
This shouldn't make intuitive sense to you since animals eat feed grown on good cropland (98% of the water footprint of animal ag) that we could eat directly, and we lose 95% of the calories when we route crops through animals.
You’re confusing my argument. I’m not arguing that meat is more efficient than plants; that by itself is obviously untrue because of the trophic efficiency loss. I’m arguing that meat uses marginal resources that would otherwise be useless for growing food. If we eliminate meat we’d eliminate a lot of marginally productive farmland that is naturally irrigated and have to replace it with industrialized irrigation farming that will be significantly more expensive and use more water and energy. Maybe if we had a global cultural reset with industrialized farming, but no one is going to be happy replacing their beef and chicken with the corn and barley and hay that those lands normally grow, much of it inedible to humans.
That paper isn’t actually debunking anything that I’m saying. If the water foot print per calorie is 20x for beef but the feed is grown with 90% of its water from rainfall, that’s not a 20x bigger footprint in a way that practically matters because most of that water is unrecoverable anyway. The water that is recoverable just makes it through the watershed.
Meat is a way to convert land that cant grow things people can or want to eat into things that people will eat. That pastureland and marginal cropland growing animal feed can’t just be converted to grow more economically productive crops like fruit and vegetables without Herculean engineering effort and tons of water and fertilizer. Instead the farming would have to stress other fertile ecosystems like the Southwest which would make the water problems worse, even if their total “footprint” is smaller. The headline that beef uses 20x more water per calorie completely ignores where that water comes from and how useful it actually is to us.
I don’t doubt that we can switch to an all plant diet as a species but people vastly underestimate the ecological and societal cost to do so.
I mean it's even simpler. Almonds are entirely non essential (many other more water efficient nuts) to the food supply and in California consume more water than the entire industrial sector, and a bit more than all residential usage (~5 million acre-feet of water).
Add a datacenter tax of 3x to water sold to datacenters and use it to improve water infrastructure all around. Water is absolutely a non-issue medium term, and is only a short term issue because we've forgotten how to modestly grow infrastructure in response to rapid changes in demand.
Thanks for the sanity!! I wish more people understood this
Ask people which they'd rather have: -no more meat and a little better AI -keep their meat and AI doesn't improve from where it is today..
I called out golf in my first comment in this thread.
If data center usage meant we didn't have enough water for agriculture I would shout that from the rooftops.
Yep--I'm agreeing that one's a good comparison to elaborate on.
Exploring how it stacks up against an essential use probably won't persuade people who perceive it as wasteful.
Growing almonds is just as essential as building an AI. Eating beef at the rate americans do is not essential. Thats where basically all the water usage is going.
Agriculture is generally essential but that doesn't mean that any specific thing done in the name of agriculture is essential.
If Americans cut their meat consumption by 10%, we would use a lot less water in agriculture and probably also live longer in general
Iran's ongoing water crisis is an example. One cause of it is unnecessary water-intensive crops that they could have imported or done without (just consume substitutes).
It's a common reasoning error to bundle up many heterogeneous things into a single label ("agriculture!") and then assign value to the label itself.
I am surprised by your analytical mistake of comparing irrigation water with data-center water usage...
They are not equivalent. Data centers primarily consume potable water, whereas irrigation uses non-potable or agricultural-grade water. Mixing the two leads to misleading conclusions on the impact.
That's a really good point - you're right, comparing data center usage to potable water usage by towns is a different and more valid comparison than comparing with water for irrigation.
They made a good point, but keep in mind that they're doing a "rules for thee, not for me" sometimes.
The same person who mentioned potable water being an important distinction also cited a report on data center water consumption that did not make the distinction (where the 628M number came from).
The problem is many data centers are in areas where water systems are supply constrained... - https://spectrum.ieee.org/ai-water-usage
This is not a distinction that your second link (that has the 628M number) was making either
> water evaporation from hydroelectric dam reservoirs in their water use calculations
The factual soundness of my argument is independent of the report quality :-) the report influences comprehension, not correctness...
The fact data centers are already having a major impact on the public water supply systems is known, by the decisions some local governments are forced to do, if you care to investigate...
https://spectrum.ieee.org/ai-water-usage
"...in some regions where data centers are concentrated—and especially in regions already facing shortages—the strain on local water systems can be significant. Bloomberg News reports that about two-thirds of U.S. data centers built since 2022 are in high water-stress areas.
In Newton County, Georgia, some proposed data centers have reportedly requested more water per day than the entire county uses daily. Officials there now face tough choices: reject new projects, require alternative water-efficient cooling systems, invest in costly infrastructure upgrades, or risk imposing water rationing on residents...."
https://www.bloomberg.com/graphics/2025-ai-impacts-data-cent...
What counts as data center water consumption here? There are many ways to arguably come up with a number.
Does it count water use for cooling only, or does it include use for the infrastructure that keeps it running (power generation, maintenance, staff use, etc.)
Is this water evaporated? Or moved from A to B and raised a few degrees.
This is the real point. Just measuring the amount of water involved makes no sense. Taking 100 liters of water from a river to cool a plant and dumping them back in a river a few degrees warmer is different from taking 100 liters from a fossil acquifer to evaporatively cool the same plant.
Humans don’t consume much water used by humans. Cows do.
A farmer is a valuable perspective but imagine asking a lumberjack about the ecological effects of deforestation, he might know more about it than an average Joe, but there's probably better people to ask for expertise?
> Honestly, we probably use more water than they do
This kind of proves my point, regardless of the actual truth in this regard, it's a terrible argument to make: availability of water starts to become a huge problem in a growing amount of places, and this statement implies the water usage of something, that in basic principle doesn't need water at all, uses comparable amount of water as farming, which strictly relies on water.
The author of the article followed the quote from the farmer with a fact-checked (this is the New Yorker) note about water usage for AI training.
I think the point here is that objecting to AI data center water use and not to say, alfalfa farming in Arizona, reads as reactive rather than principled. But more importantly, there are vast, imminent social harms from AI that get crowded out by water use discourse. IMO, the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology.
> the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology
Especially since so many anti-crypto people immediately pivoted to anti-AI. That sudden shift in priorities makes it hard to take them seriously.
On the flip side, the crypto hype machine pretty seamlessly flipped to the AI hype machine, so it makes sense the same anti crowd shifted pretty seamlessly. Given the practical applications of crypto were minimal and the externalities were mostly crime and pollution, I’m not at all surprised that many people expect the same for AI.
The anti-crypto people were correct, though. Why should we not push back when we’re seeing the same type of baseless hype that surrounded crypto being cultivated around the AI space?
They were and we should push back and yes, there is a mountain of baseless hype. But if you train your fire on the wrong thing, you risk not addressing the actual problem.
I didn't pivot - I'm still strongly anti-crypto and pretty strongly anti-AI.
But if I say "I object to AI because <list of harms> and its water use", why would you assume that I don't also object to alfalfa farming in Arizona?
Similarly, if I say "I object to the genocide in Gaza", would you assume that I don't also object to the Uyghur genocide?
This is nothing but whataboutism.
People are allowed to talk about the bad things AI does without adding a 3-page disclaimer explaining that they understand all the other bad things happening in the world at the same time.
Because your argument is more persuasive to more people if you don't expand your criticism to encompass things that are already normalized. Focus on the unique harms IMO.
No, that's not the point.
If you take a strong argument and through in an extra weak point, that just makes the whole argument less persuasive (even if that's not rational, it's how people think).
You wouldn't say the "Uyghur genocide is bad because of ... also the disposable plastic crap that those slave factories produce is terrible for the environment."
Plastic waste is bad but it's on such a different level from genocide that it's a terrible argument to make.
Adding a weak argument is a red flag for BS detectors. It's what prosecutors do to hoodwink a jury into stacking charges over a singular underlying crime.
I don't think there's a world where a water use tax is levied such that 1) it's enough for datacenters to notice and 2) doesn't immediately bankrupt all golf courses and beef production, because the water use of datacenters is just so much smaller.
We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Beef, I guess, is a popular type of food. I’m under the impression that most of us would be better off eating less meat, maybe we could tax water until beef became a special occasion meal.
I'm saying that if you taxed water enough for datacenters to notice, beef would probably become uneconomical to produce at all. Maybe a good idea! But the reason datacenters would keep operating and beef production wouldn't is that datacenters produce way more utility per gallon.
Taxes can have nuance.
You can easily write a law that looks like this: There is now a water usage tax. It applies only to water used for data-centers. It does not apply to residential use, agricultural use, or any other industrial use.
We do preferential pricing and taxing all the time. My home's power rate through the state owned utility is very different than if I consumed the exact same amount of power, but was an industrial site. I just checked and my water rate at home is also different than if I were running a datacenter. So in all actuality we already discriminate for power and water based on end use. at least where I live. Most places I have lived have different commercial and residential rates.
In other words, the price of beef can stay the same.
Yes, if you don't like datacenters, tax datacenters. If you want to do it via a tax on the water they use then I guess that works.
But the environment doesn't really care whether the water is being used by a datacenter or something else. My point is just that data centers are actually more efficient users of water compared to many less-controversial users.
>In other words, the price of beef can stay the same.
And yet, if you believe the environmentalist argument in the first place, the price of beef should go up for the damages it causes. Hence why a lot of people think the people complaining about AI are wearing an environmentalist mask, rather than having an actual care about the environment.
Pigouvian taxes are fine, but should be applied across all sources of damage.
A lot of beef is produced in such a way that taxing municipal water won't make a material difference. Even buying market rate water rights in the high desert, which already happens in beef production, is a pretty small tariff on the beef.
If it only applies to beef, fine. If it's ALL agriculture...
I can live without another datacenter - I get very little utility from "one more" - but I have to eat, generally every day..
> We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Might as well get rid of all the lawns and football fields while we’re at it.
That's a debate worth having too! And it doesn't block this AI debate from continuing nonetheless.
Water taxes should probably be regional. The price of water in the arid Southwest is much higher than toward the East coast. You might see both datacenters and beef production moving toward states like Tennessee or Kentucky.
Of course golf courses are useful as shown by the fact that people pay to use them. Perhaps you mean that you personally haven't (yet) found them useful, but you know that different people want different things. I think eating shrimp is disgusting and never eat them but I don't want to ban global shrimp production because the people it would harm are not me!
You're not wrong.
My perspective from someone who wants to understand this new AI landscape in good faith. The water issue isn't the show stopper it's presented as. It's an externality like you discuss.
And in comparison to other water usage, data centers don't match the doomsday narrative presented. I know when I see it now, I mentally discount or stop reading.
Electricity though seems to be real, at least for the area I'm in. I spent some time with ChatGPT last weekend working to model an apples:apples comparison and my area has seen a +48% increase in electric prices from 2023-2025. I modeled a typical 1,000kWh/month usage to see what that looked like in dollar terms and it's an extra $30-40/month.
Is it data centers? Partly yes, straight from the utility co's mouth: "sharply higher demand projections—driven largely by anticipated data center growth"
With FAANG money, that's immaterial. But for those who aren't, that's just one more thing that costs more today than it did yesterday.
Coming full circle, for me being concerned with AI's actual impact on the world, engaging with the facts and understanding them within competing narratives is helpful.
Not only electricity, air pollution around some datacenters too
https://www.politico.com/news/2025/05/06/elon-musk-xai-memph...
I'd love to say the air pollution issue get the attention that's currently being diverted to the water issue!
> It's an externality like you discuss.
It's not even an externality? They just pay market price for water. You can argue the market price is priced badly (e.g., maybe prices are set by the state), but that doesn't make it an externality. The benefits/costs are still accrued by (and internal to) buyer and seller.
If datacenters are getting electricity and water at a rate lower than retail (costs passed on to residents or tax payers), and factors like noise and water pollution aren't factored in, then yes there are unpriced externalities.
In what way are they not paying for it?
Update on the update: I asked Andy Masley (author of this piece https://andymasley.substack.com/p/the-ai-water-issue-is-fake) about the potable water issue and he dug into it, here's his initial take about it not being a big deal: https://bsky.app/profile/andymasley.bsky.social/post/3m5mqce...
Key quote:
> If any region has a lot of freshwater and little potable water, the best way to make potable water more available and cheaper is to introduce a new large buyer, which will give the local utility enough revenue to upgrade and expand their treatment facilities. Saying that my data is misleading because Al "only uses valuable potable water" actually gets the issue backwards: adding demand for more potable water in regions with lots of freshwater makes potable water cheaper and more abundant for everyone else per unit.
This feels like it makes a lot of assumptions:
1. That just because a region doesn’t have enough potable water to support humans and data centers, it also doesn’t have enough potable water to support the humans alone.
2. That the temporary increase in water prices due to the new demand of the data centers will provide enough revenue to upgrade its facilities
3. Even given enough revenue to upgrade its facilities, that the utility will choose to upgrade its facilities and increase demand
4. That the downsides of a temporary increase in water prices while new facilities are built is acceptable and will not cause suffering
5. Even after new facilities are built, that the cost of those facilities will be low enough and the increase in supply large enough that water prices for humans will be lower than they were originally, even with a large and wealthy new buyer on the market.
It doesn’t feel like a very strong argument to me.
Just as an example, Loudoun County has the most data center concentration in the world by a wide margin, and also has significantly lower water prices than the Virginia average. Seems like they've been able to build out huge new water capacity for data centers and that hasn't raised househould prices, or even kept them low. https://andymasley.substack.com/i/171855599/the-county-with-...
This part of his argument is not self evident or intuitive and I'm not convinced that the abstract economic model maps cleanly to the messy reality. I'm much more assuaged by the fact that it seems to cost ~$1/1000 gallons to convert fresh water into potable water.
Like, if agriculture uses fresh water and data centers use potable water, the important question is how hard it is to convert fresh water into potable water?
The answer seems to be "not very" so the difference is kind of moot
Here's his full argument: https://andymasley.substack.com/p/the-ai-water-issue-is-fake...
There's a bunch in there but this bit caught my eye in particular:
> The US public water supply uses ~40 billion gallons per day, all of this is potable. Data centers used 50 million gallons per day onsite in 2023. So their potable water usage was 0.13% of the public water supply.
That's a naive argument. Infrastructure construction timelines are typically measured in years and decades. You need to find the political will to do it and sufficient guarantees of long-term demand to justify the investment. And the work itself is often done in difficult environments, such as under major streets or on privately owned land that may already be in use.
> Infrastructure construction timelines are typically measured in years and decades.
Years isn't actually a problem. How quickly do you think datacenters are built?
It's entirely feasible to build new water infrastructure and a datacenter in 2 years in many parts of the world.
In parts that do not care much about property rights and citizens are not allowed to challenge government decisions in the courts.
If I was trying to build new infrastructure in Finland, I would add four years to the project between the planning and the construction stages. Two years for urban planning and public hearings, and another two years for the inevitable lawsuits. Other Western countries should have similar delays.
Farmland, AI data centers, and golf courses do not provide the same utility for water used. You are not making an argument against the water usage problem, you are only dismissing it.
Growing almonds uses 1.3 trillion gallons of water annually in California alone.
This is more than 4 times more than all data centers in the US combined, counting both cooling and the water used for generating their electricity.
What has more utility: Californian almonds, or all IT infrastructure in the US times 4?
Almonds are pretty cherry picked here as notorious for their high water use. Notably, we're not betting an entire economy and pouring increasing resources into almond production, either. Your example would be even more extreme if you chose crops like the corn used to feed cattle. Feeding cows alone requires 21.2 trillion gallons per year in the US.
The people advocating for sustainable usage of natural resources have already been comparing the utility of different types of agriculture for years.
Comparatively, tofu is efficient to produce in terms of land use, greenhouse gas emissions, and water use, and can be made shelf-stable.
>Almonds are pretty cherry picked here as notorious for their high water use.
If water use was such a dire issue that we needed to start cutting down on high uses of it, then we should absolutely cherry pick the high usages of it and start there. (Or we should just apply a pigouvian tax across all water use, which will naturally affect the biggest consumers of it.)
Yes, that's roughly what I said in my post. If we're doing a controlled economy and triaging for the health of the ecosystem, we'd start with feed for cattle, and almonds wouldn't be much further down on the list.
The contention with AI water use is that something like this is currently happening as local water supplies are being diverted for data-centers.
>Notably, we're not betting an entire economy and pouring increasing resources into almond production,
Excellent, that means we can save massive amounts of water by stopping almond production in the western US.
People have been sounding the alarm about excessive water diverted to almond farming for many years though, so that doesn't really help the counter-argument.
Example article from a decade ago: https://www.motherjones.com/environment/2015/01/almonds-nuts...
Aren't Californian almonds like 80% of the world's production?
Is the US AI data-centers producing 80% of the world's IT ?
I ask legitimately, I think that would already make it more apples to apples.
Also if you ask me personally, I'd rather have almonds than cloud AI compute. Imagine a future 100 years from now, we killed the almonds, never to be enjoyed ever again by future generations... Or people don't have cloud AI compute. It's personal, but I'd be more sad that I'd never get to experience the taste of an almond and all the cuisine that comes with it.
> Is the US AI data-centers producing 80% of the world's IT
You've misread it. It's not compared to AI datacenters, it's every type of datacenter, for all types of computing.
In the future scenario you've laid out it wouldn't be cloud AI compute. You wouldn't be able to use HN or send email or pay with a credit card or play video games or stream video.
But the recent complaints are specific about cloud AI compute no? And how it's driving the build out of all these new data-centers?
Depends on what the datacenters are used for.
AI has no utility.
Almonds make marzipan.
AI has way more utility than you are claiming and less utility than Sam Altman and the market would like us to believe. It’s okay to have a nuanced take.
"AI has no utility" is a pretty wild claim to make in the tail end of 2025.
Still surprised to see so many take this as such a hot claim. Is there hype, absolutely, is there value being driven also absolutely.
Whenever I see someone say AI has no utility, I'm happy that I don't have to waste time in an argument against someone out of touch with reality.
I'm more unhappy than happy, as there are plenty of points about the very real bad side of AI that are hurt by such delusional and/or disingenuous arguments.
That is, the topic is not one where I have already picked a side that I'd like to win by any means necessary. It's one where I think there are legitimate tradeoffs, and I want the strongest arguments on both sides to be heard so we get the best possible policies in the end.
I agree, but you can't win against religious people. Better spend your time talking to the rest of us.
The article made a really interesting and coherent argument, for example. That's the kind of discourse around the topic I'd like to see.
If AI is so useful, we should have a ton of data showing an increase in productivity across many fields. GDP should be skyrocketing. We haven’t seen any of this, and every time a study is conducted, it’s determined that AI is modestly useful at best.
I don't need data to know that I use it every day and offload lots of tasks to it.
It's like the smooth brains who still post "lol 6 fingers"
Well, I don't like marzipan, so both are useless? Or maybe different people find uses/utility from different things, what is trash for one person can be life saving for another, or just "better than not having it" (like you and Marzipan it seems).
ok in that case you don't need to pick on water in particular, if it has no utility at all then literally any resource use is too much, so why bother insisting that water in particular is a problem? It's pretty energy intensive, eg.
Marzipan is fun, but useful?
AI is at least as useful as marzipan.
Activated almonds create funko pops. I’d still take the data centers over the funko pops buying basedboys that almonds causes.
AI has no utility _for you_ because you live in this bubble where you are so rabidly against it you will never allow yourself to acknowledge it has non-zero utility.
[flagged]
What does it mean to “use” water? In agriculture and in data centers my understanding is that water will go back to the sky and then rain down again. It’s not gone, so at most we’re losing the energy cost to process that water.
The problem is that you take the water from the ground, and you let it evaporate, and then it returns to... Well to various places, including the ground, but the deeper you take the water from (drinking water can't be taken from the surface, and for technological reasons drinking water is used too) the more time it takes to replenish the aquifer - up to thousands of years!
Of course surface water availability can also be a serious problem.
So with the water used in datacenters. It's just a cooling loop, the output is hot water.
and water from datacenters goes where...? just disappears?
No it’s largely the same situation I think. I was drawing a distinction between agricultural use and maybe some more heavy industrial uses while the water is polluted or otherwise rendered permanently unfit for other uses.
I'll take the almonds any day.
Other people might have other preferences. Maybe we could have a price system where people can express their preferences by paying for things with money, providing more money to the product which is in greater demand?
Sure.. Except some people / companies have so much more money, they can demand impractical things and pay above-market rates for them, causing all others to scramble to live day-to-day with the distorted market.
Tried buying a GPU lately?
Right, I think a data center produces a heck of a lot more economic and human value in a year - for a lot more people - than the same amount of water used for farming or golf.
you can make a strong argument for the greater necessity of farming for survival, but not for golf...
God forbid the public has an amenity.
I mean... Food is pretty important ...
Which is why the comparison in the amount of water usage matters.
Data centers in the USA use less than a fraction of a percent of the water that's used for agriculture.
I'll start worrying about competition with water for food production when that value goes up by a multiple of about 1000.
The water intensity of American food production would be substantially less if we gave up on frivolous things like beef, which requires water vastly out of proportion to its utility. If the water numbers for datacenters seem scary then the water use numbers for the average American's beef consumption is apocalyptic.
I appreciate that you feel this way, it’ll never happen.
The US will never give up on eating meat. Full stop.
For every vegan/vegetarian in the US there are probably 25 people that feed beef products to their pets on a daily basis.
Whether people would switch off meat on their own is a separate issue. If water became scarce enough to start moving the price, then you'd absolutely see people eat less meat.
But their point does disarm the suggestion that water consumption for AI is bad because it's just for fun while meat feeds people.
Because when you eat meat, you could have eaten something far less resource intensive like tempeh. But you ate meat for reasons beyond survival. For most of us, it's because we like the taste and we're used to it.
I don't see that as having any stronger of a claim to water consumption than the things we use AI for (fun, getting work done, writing nix/k8s config) much less a claim to many times the amount of water consumption than AI data centers.
While I agree, the "meat is not sustainable" argument is literal, and evidenced in beef prices rising as beef consumption lowers over the past years. Beef is moving along the spectrum from having had been a "staple" to increasingly being a luxury.
The US never gave up eating lobster either, but many here have never had lobster and almost nobody has lobster even once a week. It's a luxury which used to be a staple.
Beef is not the only meat. Chicken is much less water intensive.
From the animal welfare perspective, there's much more suffering involved in producing a pound of chicken than a pound of beef.
That depends how sentient a chicken is: their brains are of similar complexity to the larger of these models, counting params as synapses.
Also, while I'm vegetarian going on vegan, welfare arguments are obviously not relevant in response to an assertation that Americans aren't going to give up meat, because if animal welfare was relevant then Americans would give up meat.
Well, yeah. And I'm not vegetarian either. But it's just a fact that beef production is a vastly less efficient user of water than datacenters.
> I appreciate that you feel this way, it’ll never happen. The US will never give up on eating meat. Full stop.
I don't see any signs that the US is going to give up on AI and data centers, either. (The coming AI winter notwithstanding)
For what it's worth, I've cut back quite a bit on my beef and pork consumption, and now mostly eat chicken. The environmental and ethical arguments finally got to me.
Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds.
Also a lot less meat in general. A huge part of our agriculture is feed to feed our food. We need some meat, but the current amount is excessive
> Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds. Both alfalfa and almonds contain a lot of nutrients you dont find in large enough amounts (or at all) in corn and potatoes though. And alfalfa improves the soil but fixating nitrogen. Sure almonds require large amounts of water. Maybe alfalfa does as well? And of course it depends on if they are grown for human consumption or animal.
Water usage largely depends on the context, if the water source is sustainable, and if it is freshwater.
Of course water used up will eventually evaporate, and produce rainfall in the water cycle, but unfortunately at many places "fossil" water is used up, or more water used in an area then the watershed can sustainably support.
This is a constant source of miscommunication about water usage, and that of agriculture also. It is very different to talk about the water needs to raise a cow in eg. Colorado and in Scotish highlands, but this is usually removed from the picture.
The same context should be considered for datacenters.
They are making an anti-disruption argument.
I think it's bad though to be against growth, for reasons I've described in another comment.
That is correct, AI data centers deliver far more utility per unit of water than farm/golf land.
who are you to determine the utility? we have the market for it and it has spoken.
Data center cooling towers have to use fresh rather than salt water, but they don't care about bacterial contamination or toxic traces of arsenic, antimony, and fluorine. Agriculture also has to use fresh rather than salt water. I can't think of any circumstances where water that was usable for agriculture wouldn't also be usable for cooling data centers—except when the farmer owns the water and the data center operator doesn't.
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
Not to mention that this is likely a very short-term debate. Modern AI/ML/compute has been a huge boost to fusion R&D, and tons of AI/tech capital is flowing into companies like Helion and CFS. Necessity is the mother of invention and all.
It may be a valid criticism today, but no one will be complaining about AI's environmental impact after those first few plants go live and mass production begins within the next decade. Knock on wood.
I'm skeptical about that. Helion in particular might work (https://www.youtube.com/watch?v=HlNfP3iywvI) but all the other fusion-energy companies' plans are to convert fusion energy to thermal energy and then convert the thermal energy to electrical energy in the usual way, using steam engines. That's never going to be cheaper than thermal energy powering steam engines, because thermal energy powering steam engines is part of it.
But solar energy is.
Helion's strategy is to directly convert the hot plasma's expansion into electrical energy by, basically, pushing against a powerful magnetic field. This is potentially higher efficiency than steam engines (because it's working at a higher temperature) and potentially higher power density and reliability. So it could at least theoretically get to be cheaper than steam engines. But I think it's more likely to be far more expensive for the next several decades.
I'm definitely more optimistic about Helion. It's the one that would really dramatically change the economics of power generation at scale, and they seem to have substantially engineered around major challenges like not requiring ignition and minimizing losses to Bremsstrahlung radiation. I like that they're laser-focused on launching and delivering commercial value, even if that means a bit less transparency and less focus on research demonstrations than many would prefer.
If Helion delivers Orion on anywhere close to the target timeline, it seems like they'd be primed to kick off mass production and start shipping units all over the country with minimal regulatory hurdles. With sufficiently high productive capacity, it would effectively make all other forms of power generation obsolete at utility scale. Solar would still great for small/decentralized use cases, but there wouldn't be as much reason for a power company to build new solar plants if they had the option to fill the same real estate with a bunch of Helion machines.
That being said, I'm not not optimistic about steam-based fusion power. It would be less revolutionary, but from what I understand would still be a huge advancement. My read is that it would effectively obsolete fission power and plausibly fossil fuel power as well, if not also grid-scale renewables. It would be like having SMRs with higher efficiency and no major proliferation or meltdown concerns that amplify costs. High neutron output and steam conversion add costs and complexity, but if CFS panned out and Helion didn't, I'd hardly turn my nose at that.
I could also see a world where multiple forms of fusion power pan out, and it turns out to be economical for Helion to sell tritium to other vendors like CFS rather than storing it for 12 years until it decays to He3. Maybe we'd even have colocated plants in a certain ratio where Helion provides on-site tritium generation in the exact volume needed for a D-T reactor.
https://www.helionenergy.com/articles/starting-to-build-the-... (undated) says Orion is intended to deliver 50 megawatts. If they could deliver that for under US$100 million they'd be competitive with fossil-fuel plants, assuming no fuel cost or other opex; if they could deliver it for under US$30 million they'd be competitive with solar, unless solar gets even cheaper. (It's fallen in cost by 45% over the last two years, according to https://www.solarserver.de/photovoltaik-preis-pv-modul-preis..., but almost all of that was more than a year ago; it's only fallen 10% in the last year.)
Helion's latest funding round was US$425 million, bringing total funding to over a billion, on a US$5 billion valuation. To give that money back to their investors, assuming a 40% profit margin, they'd need to sell something like 100 copies of Orion at US$30 million each, totaling 5 gigawatts. For the investors to consider them successful, it would need to be more like 2000 copies of Orion, totaling 100 gigawatts.
That's a completely plausible market size (although it's not just "all over the country", because the country they're in only has 1161 GW of generation capacity and is barely building any), so their NRE costs to date are not an impossible burden. It really depends on four things:
1. Can they hit such a low price target as US$30 million per 50-megawatt power plant? Can they ship anything for US$30 million a pop?
2. Will they be allowed to sell to China, where the customers are?
3. Will solar energy get cheaper still, forcing them to an even lower price point?
4. Will opex of the resulting reactors be manageable?
I'm not an expert on the subject, but is there an obvious bottleneck that would prevent Helion from manufacturing an arbitrarily large number of generators that scales with demand? 2000 generators doesn't seem like an overwhelmingly large number if they get paid enough to build out capacity for it, and they've already started on an assembly line manufacturing plant.
I'd expect that they probably wouldn't focus on copying Orion exactly, since Orion is just the initial PoC and will only use a single 50 MW generator. Their Nucor plant is planned to have 10x higher capacity, which seems likely to be more in line with typical deployments outside of remote areas. Opex also seems like a non-issue to me, given that the machines will be automated, the fuel will be cheap, and they won't have the same safety-critical need for strict regulation and human oversight as fission plants.
My understanding is that exporting might not be a huge hurdle to get a green light for due to the low proliferation concerns, although I wouldn't be surprised if Helion itself had some reservations around that or blocked it on designing some sort of tamper-resistant enclosure for IP protection purposes.
But I also don't see a reason why the US couldn't be a highly lucrative market for Helion, particularly given the increasing demand from AI. Solar in China today benefits from major alignment of interests and political tailwinds. Similarly, fusion could cut through partisan divides and NIMBYism that have hamstrung power buildout in the US. The left will buy in for climate benefits, the right will buy in for energy independence, both will buy in for economics + practicality, and NIMBYs will buy in for lower electric bills with minimal risks or drawbacks. Whether or not directly publicly funded, there will at least be every incentive to collectively get out of private industry's way and allow mass fusion deployment with minimal regulatory burdens.
That said, I'm sure solar will continue to get cheaper, and iron-air batteries from companies like Form Energy seem to be an extremely promising solution for high-scale storage. I also like the idea of using Prometheus Fuels to repurpose existing fossil fuel plants as storage for renewables. But there are still theoretical limits on the efficiency of solar panels, and space usage and intermittency could make it less cost-efficient than fusion in many if not most cases. We can't really make an apples-to-apples comparison between a mature technology in active high-volume production and a speculative future technology, but the latter in this case does seem to me to have a higher ceiling on its potential value over the long run.
> I'm not an expert on the subject, but is there an obvious bottleneck that would prevent Helion from manufacturing an arbitrarily large number of generators that scales with demand?
No, nothing is obvious to anyone outside the company.
> I'd expect that they probably wouldn't focus on copying Orion exactly,
Hopefully their next model will come out before another billion dollars of investment goes down the drain.
> Opex also seems like a non-issue to me, given that the machines will be automated, the fuel will be cheap,
Yes, that could happen, but what it really means is that we don't have any idea what the opex is.
> My understanding is that exporting might not be a huge hurdle to get a green light for due to the low proliferation concerns,
I think the bigger concern there would be whether China would import it.
> But I also don't see a reason why the US couldn't be a highly lucrative market for Helion,
Well, the US hasn't been a highly lucrative market for power plants for 50 years. Maybe the advent of new, radically cheaper technology could change that, but I suspect it won't; the advent of new, radically cheaper energy in the late 18th century didn't turn China or Japan into a highly lucrative market for power plants, for essentially social or political rather than technical and economic reasons.
> We can't really make an apples-to-apples comparison between a mature technology in active high-volume production and a speculative future technology, but the latter in this case does seem to me to have a higher ceiling on its potential value over the long run.
Yes, I agree. But we weren't talking about the long run, but rather about currently existing companies that need to return money to their investors in the next ten or twenty years. Breakthroughs do happen, but rarely.
Yeah, there are lots of unknowns and valid reasons to be pessimistic or optimistic. I lean toward cautious optimism based on the information that is available, but of course no one knows the future until it happens.
Just to clarify, Orion is one site, not a particular generator model. Orion working would be the green light to build and ship more generators, not a sign that they'd need to go back to the drawing board on an "Orion v2".
re: opex, we do have some idea of what it will be. We know that deuterium is cheap, we know that fusion produces low waste, we know that extensive safety protocols to protect against meltdown are not required, we know that preventing theft of fissile materials is not a security concern, and we know that these machines are substantially automated. We may not have precise balance sheets, but it's also not entirely opaque or mysterious.
I'm not sure the examples of historical China and Japan apply. They didn't invent coal power and then decide not to use it (although Japan was nevertheless fairly quick to industrialize). Fusion as of now seems likely to reach commercialization first in the US, and there's currently a lot of demand for power here with insufficient supply to meet projected growth. It's a very different market from when we were actively shipping our high-power-consumption industries overseas.
I don't agree that breakthroughs happen "rarely". I mean, we wouldn't be here in this thread without the massive breakthrough of useful LLMs, which were considered sci-fi before ChatGPT's release less than three years ago; fusion had the breakthrough of ignition at NIF just a few days after that; and we're only in the position of considering solar and batteries as a competitive energy source due to many small breakthroughs that have occurred in recent history. Breakthroughs happen all the time, but we quickly forget their significance and let them become boring.
One of the biggest problems in environmental education (imo) is the lack of notion regarding the footprint of products and services we consume daily; from the water and CO2 costs of a meal, of a heated pool, of a car, etc. It is eye-opening.
I first came across this type of info with the book "How Bad Are Bananas", from Mike Berners-Lee. I really enjoyed it, and I just saw that the new edition even includes stuff like hosting a World Cup, data centers, and space tourism!
It should give a good foundation to start talking about it.
This article goes into an ungodly amount of detail -- with receipts -- making the case that the AI water issue is fake:
https://andymasley.substack.com/p/the-ai-water-issue-is-fake
Water location matters. Is the data center in a desert with scarce potable water for locals? Or is next to a large Canadian lake, plenty of potable water, with people who want to trade something for currency so they can put avocados in their salad?
A lot of data centers are near the Columbia river, as power is cheap there thanks to hydroelectric; which flows through an arid desert-like region, but is also the largest river in the western US and it's simply impossible to pump too much water out of it.
Another issue is that you could, in principle, build data centers in places where you don't need to evaporate water to cool them. For example, you could use a closed loop water cooling system and then sink that heat into the air or into a nearby body of water. OVH's datacenter outside Montreal¹ does this, for example. You can also use low carbon energy sources to power the data center (nuclear or hydro are probably the best because their production is reliable and predictable).
Unlike most datacenters, AI datacenters being far away from the user is okay since it takes on the order of seconds to minutes for the code to run and generate a response. So, a few hundred milliseconds of latency is much more tolerable. For this reason, I think that we should pick a small number of ideal locations that have a combination of weather that permits non-sub-ambient cooling and have usable low carbon resources (either hydropower is available and plentiful, or you can build or otherwise access nuclear reactors there), and then put the bulk of this new boom there.
If you pick a place with both population and a cold climate, you could even look into using the data center's waste heat for district heating to get a small new revenue stream and offset some environmental impact.
1: https://www.youtube.com/watch?v=RFzirpvTiOo
>that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
[1] https://datacentremagazine.com/articles/reclaimed-wastewater...
Some time ago, I read the environmental impact assessment for a proposed natural gas thermal power plant, and in it they emphasized that their water usage was very low (to the point that it fit within the unused part of the water usage allowance for an already existing natural gas thermal power plant on the same site) because they used non-evaporative cooling.
What prevents data centers from using non-evaporative cooling to keep their water usage low? The water usage argument loses a lot of its relevant in that case.
Does it route the hot water back into a river?
In europe several power plants get shut down each summer because the heated water from those plants would have significant impact on the local wildlife.
> Does it route the hot water back into a river?
That particular one routed the hot water to a set of fan-cooled radiators (rejecting most of the heat into the air).
I went down that “water use” rabbit hole a month ago and basically… it’s just a bunch of circular reporting that was based on some paper published in 2023[1]. For ChatGPT 3.5 they claimed “500ml for 10-50 responses”. In 2024, Washington Post published an article that took their claim and said “519 milliliters per email”[2] but didn’t source that from the original paper… that “shocking headline” took off and got widely circulated and cited directly, treating the WaPo calculation as if it were the original research finding. Then tech blogs and advocacy sites ran with it even harder, citing each other instead of any actual research[3].
If you look at the original paper they are quite upfront with the difficulty of estimating water use. It’s not public data—in fact it’s usually a closely held trade secret, plus it’s got all kinds of other issues like you don’t know where the training happened, when it happened, what the actual cooling efficiency was, etc. The researchers were pretty clear about these limitations in the actual paper.
Basically, it’s urban legend at this point. When OpenAI’s CEO later said ChatGPT uses ~0.3ml per query, that’s roughly 100x less than the viral claims.
[1] <https://arxiv.org/abs/2304.03271> [2] <https://www.washingtonpost.com/technology/2024/09/18/energy-...> [3] <https://www.seangoedecke.com/water-impact-of-ai>/
> but if you compare data centers to farmland or even golf courses they're minuscule.
People are critical of farmland and golf courses, too. But Farmland at least has more benefit for society, so they are more vocal on how it's used.
The problem is more one of scale: a million liters of water is less than half of a single Olympic-sized swimming pool. A single acre of alfalfa typically requires 4.9 - 7.6 million liters a year for irrigation. Also, it's pretty easy to recycle the data center water, since it just has to cool and be sent back, but the irrigation water is lost to transpiration and the recycling-by-weather process.
So, even if there's no recycling, a data center that is said to consume "millions" rather than tens or hundreds of millions is probably using less than 5 acres of alfalfa in consumption, and in absolute terms, this requires only a swimming-pool or two of water per years. It's trivial.
> The problem is more one of scale:
I think the source is the bigger problem. If they take the water from sources which are already scarce, the impact will be harsh. There probably wouldn't be any complaints if they would use sewerage or saltwater from the ocean.
> Also, it's pretty easy to recycle the data center water, since it just has to cool
Cooling and returning the water is not always that simple. I don't know specifically about datacentres, but I know about wasting clean water in other areas, cooling in power plants, industry, etc. and there it can have a significant impact on the cycle. At the end it's a resource which is used at least temporary, which has impact on the whole system.
> If they take the water from sources which are already scarce, the impact will be harsh.
Surprised I had to scroll down this far to see this mentioned.
The water use argument is highly local and depends on where we are building these data centers. Are you building in the great lakes region with plenty of fresh water and no water scarcity issues (yet)? Ok fine.
But we aren't building there. We're building in Arizona, Nevada, Nebraska, Iowa putting further stress in an area that water scarcity is already an issue, or soon going to become one due to long term drought conditions. Or Texas, which already has problems with their power grid.
We're building in these locations because they're cheap. If we're going do to this, we need to not let the bottom line be the sole driving decision of data center locations. If it's not profitable to build elsewhere, don't build it until you've figured out how to make it efficient enough to where it is profitable.
> don't build it until you've figured out how to make it efficient enough to where it is profitable.
This goes against American freedom
Yes - and the water used is largely non-consumptive.
Not really. The majority of data center water withdrawal (total water input) is consumed ("lost" to evaporation etc...) with a minority of it discharged (returned in liquid form). I believe it's on the order of 3/4ths consumed, but that varies a lot by local climate and cooling technology.
There's lots of promising lower-consumption cooling options, but seems like we are not yet seeing that in a large fraction of data centers globally.
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing) 177-comment discussion on a small component of the author's argument. The author's argument is an important one that hardly hinges at all on water usage specifically, given the vast human and financial capital invested in LLM buildout so far.
Your context is a little lacking. Golf courses almost universally have retention ponds/wells/etc at the facility and recycle their water.
Only 14% use municipal water systems to draw water. https://www.usga.org/content/dam/usga/pdf/Water%20Resource%2...
"Presented by the USGA" (the United States Golf Association) gave me a wry chuckle there.
That said, here are the relevant numbers from that 2012 article in full:
> Most 18-hole golf facilities utilize surface waters (ponds, lakes) or on-site irrigation wells. Approximately 14 percent of golf facilities use water from a public municipal source and approximately 12 percent use recycled water as a source for irrigation.
> Specific water sources for 18-hole courses as indicated by participants are noted below:
> 52 percent use water from ponds or lakes.
> 46 percent use water from on-site wells.
> 17 percent use water from rivers, streams and creeks.
> 14 percent use water from municipal water systems.
> 12 percent use recycled water for irrigation.
It depends on the region. In dryer regions most golf courses are already on greywater system. Irrigation mainly targets fairways, teeboxes, and greens, while the rough is allowed to go to hell in the dryer months.
Some golf courses also are designed to spread floodwater and protect surrounding development. Golf is sort of a happy bonus on top of that.
Private courses are usually far better irrigated than their public counterparts where even the fairway is liable to be left to dry out.
I did some napkin math on data center water usage for a 500MW data center in the Columbia River valley.
It uses as much water per year as 200 acres of alfalfa in California’s Central Valley. There are around 1M acres of alfalfa growing in California.
2.5MW of data center capacity is roughly equal to 1 acre of irrigated alfalfa in water usage. If you’re pulling fossil aquifer water, open loop evaporative cooling may not be the best idea, but there are plenty of places east of 100 degrees west in the US that have virtually ‘unlimited’ water where cooling towers are a great idea since they almost double the COP of a chilled water system.
On the water front, in my area agriculture flood irrigates with pristine aquifer water, while surface water gets treated and dumped into the drinking supply. This is due to the economics of historic water rights.
The AI water usage aspect is pretty clearly a lie and a gross misunderstanding at best https://open.substack.com/pub/andymasley/p/the-ai-water-issu... There are dozens of other things that use we use everyday that have a larger impact. I think there a real concerns here but the water usage argument is a poor one
Water and power are local issues. And data center use of water and power is already, currently having local impact on politics. I saw ads about it during the election cycle that just concluded. Candidates had to answer questions about it at debates and in interviews.
People are using these arguments for the simple reason that they demonstrably resonate with average people who live near data centers.
They probably don’t resonate with people who have plenty of income and/or do not compete with data centers locally for resources.
The water argument rings a bit hollow for me not due to whataboutism but more that there's an assumption that I know what "using" water means, which I am not sure I do. I suspect many people have even less of an idea than I do so we're all kind of guessing and therefore going to guess in ways favorable to our initial position whatever that is.
Perhaps this is the point, maybe the political math is that more people than not will assume that using water means it's not available for others, or somehow destroyed, or polluted, or whatever. AFAIK they use it for cooling so it's basically thermal pollution which TBH doesn't trigger me the same way that chemical pollution would. I don't want 80c water sterilizing my local ecosystem, but I would guess that warmer, untreated water could still be used for farming and irrigation. Maybe I'm wrong, so if the water angle is a bigger deal than it seems then some education is in order.
If water is just used for cooling, and the output is hotter water, then it's not really "used" at all. Maybe it needs to be cooled to ambient and filtered before someone can use it, but it's still there.
If it was being used for evaporative cooling then the argument would be stronger. But I don't think it is - not least because most data centres don't have massive evaporative cooling towers.
Even then, whether we consider it a bad thing or not depends on the location. If the data centre was located in an area with lots of water, it's not some great loss that it's being evaporated. If it's located in a desert then it obviously is.
If it was evaporative, the amounts would be much less.
Imnot sure why you're saying it would be less? All sources I can find say that evaporative cooling is a tradeoff of more water for less power.
Just from a physics standpoint.
If you discharge water into a river, there are environmental limits to the outlet temperature (this is a good thing btw). The water can't be very hot. That means you need to pump a large volume of water through because you can only put a small amount of energy into each kg of water.
If you evaporate the water on the other hand, not only is there no temperature limit but it also absorbs the latent heat of vaporisation. The downside is it's a lot more complex and also the water is truly consumed rather than just warming it up.
Put that way, any electricity usage will have some "water usage" as power plants turn up their output (and the cooling pumps) slightly. And that's not even mentioning hydroelectric plants!
Water can range from serious concern to NBD depending on where the data center is located, where the water is coming from, and the specific details of how the data center's cooling systems are built.
To say that it's never an issue is disingenuous.
Additionally one could image a data center built in a place with a surplus of generating capacity. But in most cases, it has a big impact on the local grid or a big impact on air quality if they bring in a bunch of gas turbines.
I think the water use arguments are relevant, particularly in regions of the world and US (CA) where potable water is scarce, but land and electricity are available .
NYT article gift link where people reported wells ran dry after data centers moved in. : 'From Mexico to Ireland, Fury Mounts Over a Global A.I. Frenzy' https://www.nytimes.com/2025/10/20/technology/ai-data-center...
From https://www.eesi.org/articles/view/data-centers-and-water-co... , I understand there are two types of cooling with water in DCs, open-loop that's simple but water-intensive, and closed-loop that's expensive but efficient.
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
The reported case about water wells running dry had to do with issues in construction rather than anything about the data center's regular operation:
> But the reason their taps ran dry (which the article itself says) was entirely because of sediment buildup in groundwater from construction. It had nothing to do with the data center’s normal operations (it hadn’t begun operating yet, and doesn’t even draw from local groundwater). The residents were wronged by Meta here and deserve compensation, but this is not an example of a data center’s water demand harming a local population.
https://andymasley.substack.com/p/the-ai-water-issue-is-fake...
The nice thing about the data center water usage panic is that whenever someone appeals to it, I immediately know that either they haven't done their homework or they're arguing in bad faith.
I suppose instead we can talk about people's 401k's being risked in a market propped up by the AI bubble.
We really need to stop tying people's retirement to the market. I've already lost ground due to 2008, and COVID, and while I was young, I lived through my parents suffering through dotcom as well.
It's long past time we have a universal public pension, funded at least in part with a progressive wealth tax, or least go back to private defined benefit pensions to where the employer or pension fund bears the risk rather than the individual.
Supplement it with market speculation if you want, but we need something guaranteed for everyone that's a little more robust and provides a better living than the paltry social security we have now.
Absolutely.
> data centers often use potable drinking water
hmm why exactly? mineral content?
Yeah I think it's to avoid mineral buildup on the cooling equipment which would then need to be replaced more often.
skip water discussion because it's just irrelevant. If you can debunk AGI then of course we should stop spending trillions on it. If you can't debunk AGI then water usage is just a nonfactor.
Arguments in isolation are usually poor. The water usage arguments usually (always?) comes along with a bunch of other arguments, including power consumption, workers rights, consumer protection, public safety, enshittifcation, etc.
When evaluating the economical cost or morality of a thing, (just like when training a machine learning model) the more data you consider the more accurate the result (although just like statistical modelling it is worth to be wary of overfitting).
Why can't we use non-potable water for these data centers too?
[flagged]
I'm personally excited for when the AGI-nauts start trotting out figures like...
> An H100 on low-carbon grid is only about 1–2% of one US person’s total daily footprint!
The real culprit is humans after all.
Humans have been measuring between human only vs augmented labor for literal centuries.
Frederick Taylor literally invented the process you describe in his “principles of scientific management”
This is the entire focus of the Toyota automation model.
The consistent empirical pattern is:
Machine-only systems outperform humans on narrow, formalizable tasks.
Human-machine hybrid systems outperform both on robustness, yieldjng higher success probability
Good enough?
I was making a joke.
>Tip for AI skeptics
Assumptions you are making:
- AI = transformer ANNs
- People sceptical of transformer ANNs directly leading to AGI within any reasonable period are also sceptical of transformer ANNs directly leading to AGI any time in the far future
This kind of generalisations don't help you as the huge number of comments underneath yours likely shows
I don't think anyone who read my comment here misunderstood my usage of the term "AI skeptic" as applying to any form of machine learning as opposed to modern generative AI.
What you think it is and what it is are 2 different things. You are once more making assumptions.
What assumption did I make here?
Fine, fine: get rid of golf courses too.
As for food production; that might be important? IDK, I am not a silicon "intelligence" so what do I know? Also, I have to "eat". Wouldn't it be a wonderful world if we can just replace ourselves, so that agriculture is unnecessary, and we can devote all that water to AGI.
TIL that the true arc of humanity is to replace itself!
See comment here: https://news.ycombinator.com/item?id=45927268
Given the difference in water usage, more data centers does not mean less water for agriculture in any meaningful way.
If you genuinely want to save water you should celebrate any time an acre of farm land is converted into an acre of data center - all the more water for the other farms!
the value of datacenters is dubious. the value of agriculture, less so.
I make use of AI in my farming operation. Now what?
Yeah, but the AI you use in your farming operation is running on that tiny little box behind the buddy seat.
That too, but I, more specifically, meant LLMs operating in someone else's datacenter here.
Once again, the key thing here is to ask how MUCH value we get per liter of water used.
If data centers and farms used the same amount of water we should absolutely be talking about their comparative value to society, and farms would win.
Farms use thousands of times more water than data centers.
Once again, you are ignoring my (implied) argument:
Humans NEED food, the output of agriculture. Humans do not NEED any of LLMs' outputs.
Once everyone is fed, then we can talk about water usage for LLMs.
Farms already produce more than enough food to feed everyone (and, indeed, the excess is a feature because food security is really important). The reason not everyone is fed is not due to needing to divide water resources between farms and other uses.
Going only by the effective need of humans is a bad argument. A lot of farmers wouldn't survive without subsidies and are not vital to our food supply.
Stop eating beef. With the water saved we can grow enough food for any realistic human population. Ok we solved this one. Or do humans NEED burgers as well? We can already feed all people, any starvation is strictly a political problem not a food existing on the planet problem
We produce enough food for everyone already, and then waste a huge amount of it. Our food problem isn't about producing more, it's about distributing what we have.
fine so do the same performative activism for
1. netflix
2. gmail
3. hackernews
4. discord
5. gaming
Yes, it is worthwhile to ask how much value we get.
And a whole bunch of us are saying we don't see the value in all these datacenters being built and run at full power to do training and inference 24/7, but you just keep ignoring or dismissing that.
It is absolutely possible that generative AI provides some value. That is not the same thing as saying that it provides enough value to justify all of the resources being expended on it.
The fact that the amount of water it uses is a fraction of what is used by agriculture—which is both one of the most important uses humans can put water to, as well as, AIUI, by far the single largest use of water in the world—is not a strong argument that its water usage should be ignored.
That's a fair point. For people already finding great utility in GenAI it seems like a foregone conclusion, but for others it can be understandably frustrating that people take it as an article of faith that "AI is worth it."
Disclosure, I'm very much in the former camp, but I try to ground myself with broader empirical evidence. I've found an increasing amount of empirical evidence that GenAI is providing value more than commensurate with its costs.
Largely the studies I've looked at focus on productivity boosts, I guess because that is very easy to tie to economic impact. This recent thread has some relevant sources and extremely rough numbers, but the outcome seems to be that for a 1% increase in datacenter usage we may have gotten a 1.2% boost in national productivity: https://news.ycombinator.com/item?id=45794907
Now, this absolutely not a rigorous analysis, but could be a good starting point for how to think about these tradeoffs.
it is also a fraction of golf courses which you again ignore. this is just typical "don't do anything!!" ism. there's no argument here.. even if data centres used .00001 millilitre of water you would say the same thing.
Oh, I think golf courses shouldn't exist. They're awful in a number of ways. You want to play golf? VR or minigolf.
But (as I pointed out elsewhere in this discussion [0]) why should I have to mention everything that uses water in a way I think is detrimental in order to be allowed to talk about why I think this thing uses water in a way that is detrimental?
[0] https://news.ycombinator.com/item?id=45927558
if you are at the point where you have to suggest golf courses shouldn't exist then sorry we are completely on different page here.
your logic will extend to just doing nothing at all because it will take resources from other things that you value.
Why would the logic extend there? If golf courses really do use a lot of water, energy, etc, and they largely benefit a small subset of people, it doesn’t make a ton of sense for them to exist.
Of course making the world a better place for everyone is very against western and American exceptionalism and freedom.
How much water do you think AI data centers use as a percentage of golf courses in the USA?
Complaining about someone else using an "ism" during a literal "whataboutism" is rich
According to this logic the ideal situation is when there are no farms anymore because then each (out of zero) farm gets maximum water.
Eventually people stop building more data centers as food becomes scarce and expensive, and farms become the hot new thing for the stock market, cereal entrepreneurs become the new celebrities and so on. Elon Husk, cereal magnate.
I was just pointing out the ridiculousness of the argument.
> sound scary when presented without context
It's not about it being scary, its about it being a gigantic, stupid waste of water, and for what? So that lazy executives and managers can generate their shitty emails they used to have their comms person write for them, so that students can cheat on their homework, or so degens can generate a video of MLK dancing to rap? Because thats the majority of the common usage at this point and creating the demand for all these datacenters. If it was just for us devs and researchers, you wouldn't need this many.
Whether it's a "gigantic" waste of water depends on what those figures mean. It's very important to understand if 25 million liters of water per year is a gigantic number or not.
For comparison it's about 10 olympic-sized swimming pools worth of water, doesn't seem very significant to me. Unless you're going to tell people they're not allowed swimming pools any more because swimming doesn't produce enough utility?
And at any rate, water doesn't get used up! It evaporates and returns to the sky to rain down again somewhere else, it's the most renewable resource in the entire world.
If only millions of people suffering from lack of water knew this.
Would we be sending that water to those millions of people instead?
If you redistributed this water to a million people suffering from lack of water, they'ed get about 2 shot glasses worth per day.
Yea but it's not like those people have never seen water. And yet it's not so simple, that you can use water but the water eventually comes back to you. There is a hell lot more nuance to this.
Its not gigantic and its not a waste. Brainrot creates massive economic value that can be used to pay people for products you are more happy to consume.
And also, none of those current use cases are a real benefit to society, outside of maybe research cases.
The only benefit is to the already wealthy owner class that is itching to not have to pay for employees anymore because it impacts their bottom line (payroll is typically the largest expense).
It's not like we are making robots to automate agriculture and manufacturing to move toward a post scarcity, moneyless society, which would have real benefits. No, instead we have AI companies hyping up a product whose purpose (according to them) is so that already wealthy people can hoard more wealth and not have to pay for employees. It's promising to take away a large portion of the only high-paying jobs we have left for the average person without an advanced degree.
Me being able to write software a little faster, without hiring a junior, is a net negative to society rather than a benefit.
You appear to be arguing against using technology to boost human efficiency on a forum full of software engineers who've dedicated their careers to building software that makes humans more efficient.
If we aren't doing that then why are we building software?
Because the stated goal of generative AI is not to make an individual more efficient, it's to replace that individual all together and completely eliminate the bottom rungs of the professional career ladder.
Historically software that made humans more efficient resulted in empowerment for the individual, and also created a need for new skilled roles. Efficiency gains were reinvested into the labor market. More people could enter into higher paying work.
With generative AI, if these companies achieve their stated goals, what happens to the wealth generated by the efficiency?
If we automate agriculture and manufacturing, the gain is distributed as post-scarciaty wealth to everyone.
If we automate the last few remaining white-collar jobs that pay a living wage, the gain is captured entirely by the capital owners & investors via elimination of payroll, while society only loses one of its last high-paying ladders for upward mobility.
Nobody lost their career because we built a faster operating system or a better compiler. With generative AI's stated goals, any efficiency gains are exclusively for those at the very top, while everyone else gets screwed.
Now, I'll concede and say, that's not the AI companies' fault. I'm not saying we shouldn't magically stop developing this technology, but we absolutely need our governments to start thinking about the ramifications it can have and start seriously considering things like UBI to be prepared for when the bottom falls out of the labor market.
Thanks, that's a well argued comment.
I'm not a fan of of the "replace workers with AI" thing myself - I'm much more excited about AI as augmentation for existing workers so they can take on more challenging tasks.
Does the future productivity growth that would have been gained later (due to more junior engineers not entering the field) outweigh the AI gains?
If it's just the little productivity boost now, I think it's a net negative if hiring trends continue.
I think it's a discussion to be had but talent pool is a tragedy of the commons situation.
Seems the problem is the revealed preference of the normies, rather than the technology itself.
I think the water usage argument can be pertinent depending on the context.
https://www.bbc.com/news/articles/cx2ngz7ep1eo
https://www.theguardian.com/technology/2025/nov/10/data-cent...
https://www.reuters.com/article/technology/feature-in-latin-...
That BBC story is a great example of what I'm talking about here:
> A small data centre using this type of cooling can use around 25.5 million litres of water per year. [...]
> For the fiscal year 2025, [Microsoft's] Querétaro sites used 40 million litres of water, it added.
> That's still a lot of water. And if you look at overall consumption at the biggest data centre owners then the numbers are huge.
That's not credible reporting because it makes no effort at all to help the reader understand the magnitude of those figures.
"40 million litres of water" is NOT "a lot of water". As far as I can tell that's about the same annual water usage as a 24 acre soybean field.
I agree that those numbers can seem huge without proper context.
For me, that BBC story, and the others, illustrates a trend; tech giants installing themselves in ressource-strained areas, while promoting their development as drivers of economic growth.
Yes, a 24 acre soybean field uses a lot of water.
And an average US soybean farm has 312 acres (13x larger than 24 acres): http://www.ers.usda.gov/topics/crops/soybeans-and-oil-crops/...
Which means that in 2025 Microsoft's Querétaro sites used 1/13th of a typical US soybean farm's annual amount of water.
It's a lot of water for AI waifus and videos of Trump shitbombing people who dare oppose him.
It's not a lot of water for AI weather modeling to ensure the soybean crops throughout the country are adequately watered and maximize yield.
We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible.
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
In addition to being hard, we should ask if something is useful or if the benefits outweigh the harms.
It's hard to see benefits from AI systems, AGI or otherwise. It doesn't seem to produce anything that improves human happiness or general well being.
I think you just lack imagination. I can imagine a lot of benefits of AGI or near-AGI systems.
The issue is how you can also easily imagine a lot of negative.
Neither those imagining positives or those imagining negatives can really prove to the other that they're right. This becomes the source of the conflict.
If you predicted the negative outcomes, you'd also oppose it. If you believe the outcomes will be positive, you'd be a proponent.
Now it also happens that a lot of those predicting positive outcomes are currently making money off the development of AI and so have an investment in its continued enablement. Which muddies things even more, because it's an obvious bias.
I agree. The benefits are just not well distributed.
The benefits of technology rarely are at first.
> I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.
This is not a data center problem but a policy one of the last 20+ years.
We have not been building and extending energy sources, instead from some time around 2000 we've been steady stating and increasing efficiency. That's perfectly fine, but if anything shows up and uses more power, that puts us in a losing game. We should have trashed our coal power plants forever ago, and looked at more things like nuclear and now solar and wind. Instead we are attempting to play catch up with a troglodyte foot on our face that is yelling 'burn more coal'.
Don't blame data centers, instead blame politicians that would blame culture wars and the closest minority for the rest of our problems instead of actually working on public goods.
The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
I'm an atheist too. I grew up in the church, rejected it in my teens. The problem with organized religion was the "organized" part -- the centralized, inflexible human authority.
I'm increasingly convinced that spirituality is a vital part of the human experience and we should embrace it, not reject it. If you try to banish basic human impulses, they just resurface in worse, unexpected forms somewhere else.
We all need ways to find deep connection with other humans and the universe around us. We need basic moral principles to operate on. I think most atheists like myself have quietly found this or are in the process of finding this, but it's ok to say it out loud.
For me it means meditation, frugality, and strict guidelines on how I treat others. That's like a religion, I guess. But that's OK. I embrace it. By owning it and naming it, you have mastery over it.
Thanks for saying this. I have a similar situation: " I grew up in the church, rejected it in my teens."
I've come to conclude that the problem isn't "religion" in the abstract, it's that the current institutions are unhealthy and sometimes toxic. IE, it's easier to be an atheist, even in name only, when one realizes that the religion they participated in as a child has more problems than benefits.
I sometimes wonder how people like us could create institutions that replace religions, yet generally agree with our beliefs and needs.
---
BTW: I've seen plenty of toxic behavior from atheists too. Aggressive evangelizing ones' beliefs, (or lack thereof,) IMO is a toxic behavior.
This is an odd perspective to me. I'm an atheist because I don't find the truth claims of theistic religions convincing. Whether or not the centralized structure of modern religion is problematic is tangential to that
Centralized religious authority has prevented religions from evolving or being replaced by new ones. They haven't gracefully let go of their beliefs about the universe we now know are false.
The EA / Rationalist / AI Safety crowd tend to think they can overcome these impulses
> The problem with organized religion was the "organized" part
What on earth is "unorganized" religion? How is such a concept distinct from the concepts of "worldview" or "ideology"? When people use the term, I assume automatically they're referring to abrahamic faiths—especially in contrast to "atheism".
It just means a religion with a formal structure and leadership behind it. The term points to the institution and its influence, not the beliefs themselves. An unorganized religion is basically personal or informal practice without a central authority, like an individual or essentially a book club.
> An unorganized religion is basically personal or informal practice without a central authority, like an individual or essentially a book club.
Right, but I don't know what is referred to by "practice" that is at all distinct from "worldview" or "ideology".
If the way you practice your religion is standardized by an authority across several churches, it is organized religion. For example: Catholic sects and Mormons are organized faiths. They have manuals for the priests to follow and you can go to the same one elsewhere and get mostly the same experience. Some small churches localized to a city are also organized. Islam sects in the Middle east are usually organized between Sunni and Shia. To my knowledge, Islam is not organized in the USA even though the Imam might align with a sect, because there is no authority they report to or strict standard for their patrons. Most Protestant churches are unorganized, and non-denominational are almost always unorganized because they are one-offs.
This is my informal understanding; I am not a religious scholar
I didn't say we need "unorganized religion," I said the problem with organization religion is the organization.
We need more spirituality.
Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations? Those situations are close in other aspects too: we have a physical system (the sun, the brain), whose functionality we try to replicate technologically.
> Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations?
Yes. And the people talking about "bringing the power of the Sun to Earth" are usually either full of shit or expect so little from their audience that they are effectively talking shit (or are making clickbait, what's the same). (And yes, there's a lot of fraud around fusion too.)
The people doing actual work don't use phrases like that in normal conversation.
the way the pro nuclear crowd talk you might think they are a persecuted religion actually
Or over-the-top proponents of clean intermittent energy sources. Mountains of batteries and lakes of pumped storage for the Sun and Wind god.
You don't even have to go as far as fusion reactors. Nuclear bombs are real, and we know they work.
But surely, anyone who's talking about atomic weapons must be invoking religious imagery and the old myths of divine retribution! They can't be talking about an actual technology capable of vaporizing cities and burning people into the walls as shadows - what a ridiculous, impossible notion would that be! "World War 3" is just a good old end-of-the-world myth, the kind of myth that exists in many religions, but given a new coat of paint.
And Hiroshima and Nagasaki? It's the story of Sodom and Gomorrah, now retold for the new age. You have to be a doomsday cultist to believe that something like this could actually happen!
Emmanuel Todd has been going on about "zombie" and "zero" religion in a way that is really resonating with me.
If I understand his idea correctly, these societies that were developed with a religious justification, and a huge religious component, are of course losing it in the scientific age. The first stage they go through is "zombie religion" where people don't pretend to believe in the religion any more, but still insist that they share all of its values, and often become even more fanatical in the functions that the old religion served. The second stage is "zero religion" where both the belief and the functions are gone, and all that's left is a religion shaped hole that is filled with nihilism: the strong preying on the weak, self-indulgence, and an elite retreat into often paranoid fantasy.
These stages are shaped by the particular religion that disappeared, so the Zero Catholicisms aren't the same as Zero Protestantisms aren't the same as the Zero Islams. Science, being about what works rather than why you should be doing anything, simply didn't fill up these holes that once held morality and justification. For him, it seems, the Western world is primarily in a moral crisis, and we're seeing it in the mental decay of an elite that doesn't have to justify itself to anyone, ever (after religion has died.)
Personally, I can also see this in the deep desire of some people to obey AI, but I can't see it being fruitful at all. "Because the AI said so" is not particularly inspiring or ecstatic. It's just an extension of middle-class materialistic money as grace and job as devotion, which is notoriously unfulfilling. Will AI help you succeed if it can't tell you what it means to succeed?
> The nineteenth century decided to have no religious authority. The twentieth century seems disposed to have any religious authority.
An article in Illustrated London News, April 26, 1924 by G. K. Chesterton
I've seen more religious language from AGI skeptics than believers. I kind of think AGI will happen on the basis of being able to think / process data like a human brain which I don't see as unlikely. The skeptics will say AGI is trying to build god and so not happening, but that's a strawman argument really.
The A.I. economic bubble is a mad scramble to ride the crest of a wave of stock-pumping expectations before the inevitable collapse and dump. Trillions of dollars of "value" from bloviating promises. It's worthy of a new chapter in Mackay's Extraordinary Popular Delusions and the Madness of Crowds. [1]
> religious impulses don't die with the withering of religion
Religions have of course come and gone throughout human history. The preceding deities, temples, and artwork are called mythology by people inside today's temples of fervour.
But let's be clear, disparate local tribal practices and beliefs are only formalised by a power structure for the masked purposes of the power structure.
What springs eternal is the maintenance of control in political and tribal hierarchies.
> A desire for a totalizing solution to all woes
The fact that our species exhibits astonishing credulity is illustrated throughout history to the present day, not just in religious activities but in every context of economic scams and demagoguery.
[1] _ https://en.wikipedia.org/wiki/Extraordinary_Popular_Delusion...
The thing about bubbles is that it's devilishly difficult to tell the difference between a total sham and a genuine regime shift while it's happening because the hype level is similar for both.
Sure, it certainly could be difficult. But this is often because of information assymetry — the purveyors are making big claims about the dragon in their garage. [1]
If your intuition told you that there ought to be some evidence for grand claims, you would be right to be suspicious of the continuing absence of suitable evidence.
[1] _ http://people.whitman.edu/~herbrawt/classes/110/Sagan.pdf
i think a solution to all woes should be desired
People always create god, even if they claim not to believe in it. The rise of belief in conspiracy theories is a form of this (imagining an all powerful entity behind every random event), as is the belief in AGI. It's not a totalizing solution to all woes. It's just a way to convince oneself that the world is not random, and is therefore predictable, which makes us feel safer. That, after all, is what we are - prediction machines.
The existential dread from uncertainty is so easily exploited too, and the root cause for many of societies woes. I wonder what the antidote is, or if there is one.
It's just a scam, plain and simple. Some scams can go on for a very long time if you let the scammers run society.
Any technically superior solution needs to have a built in scam otherwise most followers will ignore it and the scammers won't have incentive to prosthelytize, e.g. rusts' safety scam.
The idea of replicating a consciousness/intelligence in a computer seems to fall apart even under materialist/atheist assumptions: what we experience as consciousness is a product of a vast number of biological systems, not just neurons firing or words spoken/thought. Even considering something as basic as how fundamental bodily movement is to mental development, or how hormones influence mood ultimately influencing thought, how could anyone ever hope to to replicate such things via software in a way that "clicks" to add up to consciousness?
Conflating consciousness and intelligence is going to hopelessly confuse any attempt to understand if or when a machine might achieve either.
(I think there's no reasonable definition of intelligence under which LLMs don't possess some, setting aside arguments about quantity. Whether they have or in principle could have any form of consciousness is much more mysterious -- how would we tell?)
Defining machine consciousness is indeed mysterious, at the end of the day it ultimately depends on how much faith one puts in science fiction rather than an objective measure.
Seems like a philosophy question, with maybe some input from neuroscience and ML interpretability. I'm not sure what faith in science fiction has to do with it.
I don't see a strong argument here. Are you saying there is a level of complexity involved in biological systems that can not be simulated? And if so, who says sufficient approximations and abstractions aren't enough to simulate the emergent behavior of said systems?
We can simulate weather (poorly) without modeling every hydrogen atom interaction.
The argument is about causation or generation, not simulation. Of course we can simulate just about anything, I could write a program that just prints "Hello, I'm a conscious being!" instead of "Hello, World!".
The weather example is a good one: you can run a program that simulates the weather in the same way my program above (and LLMs in general) simulate consciousness, but no one would say the program is _causing_ weather in any sense.
Of course, it's entirely possible that more and more people will be convinced AI is generating consciousness, especially when tricks like voice or video chat with the models are employed, but that doesn't mean that the machine is actually conscious in the same way a human body empirically already is.
>but that doesn't mean that the machine is actually conscious in the same way a human body empirically
Does it matter? Is a dog/cow/bird/lizard conscious in the same way a human is? We're built from the same basic parts, and yet humans seem to have a higher state of consciousness than other animals around us.
For example the definition of the word conscious is
>aware of and responding to one's surroundings; awake.
I'll give that we likely mean this in a general sense, but I'd say we're pretty close to this with machines. They can observe the real world with sensors of different types, and then either directly compute, or use neural nets to make generalized decisions on what is occurring around them, then proceed to act on those observations.
If you simulate rainy weather, does anything get wet?
(Not my original quote, but can't remember right now where I read it.)
It's similar asking about whether silicon computers performing intelligent tasks is "conscious".
I guess it depends, can you tell the difference between a weather simulation and the actual world?
Can you?
You have weather readouts. One set is from a weather simulation - a simulated planet with simulated climate. Another is real recordings from the same place at the same planet, taken by real weather monitoring probes. They have the same starting point, but diverge over time.
Which one is real though? Would you be able tell?
They're not asking about telling the difference in collected data sets, data sets aren't weather.
The question is can you tell the difference between the rain you see outside your window, and some representation of a simulated environment where the computer says "It's raining here in this simulated environment". The implied answer is of course, one is water falling from the sky and one is a machine.
>The implied answer is of course, one is water falling from the sky and one is a machine.
Lets say you're in a room a good distance from the window. Suddenly you hear what sounds like thunder and rain falling. From a distance it appears that it's raining outside.
Is the rain real? Or is it simulated on a screen well enough you can't tell?
You have input output devices just like a computer. They don't see reality, they filter out huge amounts of data and your brain just interprets it. If our machines get good enough we may be able to blast signals directly to the brain that say it's raining and the brain wouldn't have any idea if it was simulated or not. Much in the same way it feels like we exist and not a 3d hologram an infinite distance away (or whatever other weirdness physics may or could do).
So how do we get from "machines may get really good at faking things to human perception" to "machines themselves can have human-like (or 'better') perception"?
The question is so-called consciousness arising out of machines, not machines deceiving human consciousness.
Even then, that deception still doesn't prove equivalency of simulated and real things, unless we're adopting an extreme and self-contradictory subjectivist epistemology.
People seem to way overcomplicate consciousness, especially in machines.
Where does a running video game exist? It's a simulation in the hardware. Where is the consciousness in a human brain, again it's electrical signals in our brain.
At the end of the day a system is what it does. Once it starts simulating the human mind in ways that appear human then we're at the point of saying a plane has to flap its wings or it's not flying.
You can't look at the "real weather" though. You can only look at the outputs. That's the constraint. Good luck and have fun.
A human brain is a big pile of jellied meat spread. An LLM is a big pile of weights strung together by matrix math. Neither looks "intelligent". Neither is interpretable. The most reliable way we have to compare the two is by comparing the outputs.
You can't drill a hole in one of those and see something that makes you go "oh, it's this one, this one is the Real Intelligence, the other is fake". No easy out for you. You'll have to do it the hard way.
Even granting all of your unfounded assertions; "the output" of one is the rain you see outside, "the output" of the other is a series of notches on a hard drive (or the SSD equivalent, or something in RAM, etc.) that's then represented by pixels on a screen.
The difference between those two things (water and a computer) is plain, unless we want to depart into the territory of questioning whether that perception is accurate (after all, what "output" led us to believe that "jellied meat spread" can really "perceive" anything?), but then "the output" ceases to be any kind of meaningful measure at all.
there is no "real weather". the rain is the weather. the map is not the territory. these are very simple concepts, idk why we need to reevaluate them because we all of a sudden got really good at text synthesis
Everyone's a practical empiricist until our cherished science fiction worldview is called into question, then all of a sudden it's radical skepticism and "How can anyone really know anything, man?"
You experience everything through digital signals. I dont see why those same signals cant be simulated. You are only experiencing the signal your skin sends to tell you there is rain, you dont actually need skin to experience that signal.
Bundling up consciousness with intelligence is a big assumption, as is the assumption that panpsychism is incorrect. You may be right on both counts, but you can't just make those two assumptions as a foregone conclusion.
AGI won't replicate our experience.
But it could be more powerful than us.
Honestly replicating our experience would be rather wasteful. Much like making planes the same way birds work to carry cargo.
Elon thinking Demis is the evil supervillain is hilariously backward and a mirror image of the reality.
That one struck me as... weird people on both ends. But this is Musk, who is deep into the Roko's Basilisk idea [0] (in fact, supposedly he and Grimes bonded over that) where AGI is inevitable, AGI will dominate like the Matrix and Skynet, and anyone that didn't work hard to make AGI a reality will be yote in the Torment Nexus.
That is, if you don't build the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus, someone else will and you'll be punished for not building it.
[0] https://en.wikipedia.org/wiki/Roko%27s_basilisk
It's never been explained to me why a god like AI would care one way or another whether people tried to bring it into being or not. I mean the AGI exists now, hurting the people that didn't work hard enough to bring it into existence won't benefit the AGI in any way.
God-like AI feels like we're really overestimating how tough the problem "Destroy humans" would be and so we've imagined this very impressive reason it could happen when actually we're fragile and might more or less blow away by mistake.
It seems we're much more likely to accidentally build something dumb that kills us via an unanticipated side effect. Like actual Weaboo but for society not just one business. AI helps Coca-Cola develop a new beverage that initially seems very popular and cheap so it's quickly the world's top selling drink - and then we realise, too late, that it's actually extremely addictive and withdrawal induces violent rage. Oh dear. That sort of thing.
Hence why I like using the paperclip maximizer as an example. It's not out to get you, you getting got is just an unintentional side effect. Kind of like global warming. Nobody wants the earth to burn up, they just want to give up transportation/heating/whatever else a lot less.
...or, depending on your particular version of Roko's Basilisk (in particular, versions that assume AGI will not be achieved in "your" lifetime), it will punish not you, yourself, but a myriad of simulations of you.
Won't someone think of the poor simulations??
"From my point of view the Jedi are evil!" comes to mind.
Why not both.
On the other hand we have DeepMind / Demis Hassabis, delivering:
* AlphaFold - SotA protein folding
* AlphaEvolve + other stuff accelerating research mathematics: https://arxiv.org/abs/2511.02864
* "An AI system to help scientists write expert-level empirical software" - demonstrating SotA results for many kinds of scientific software
So what's the "fantasy" here, the actual lab delivering results or a sob story about "data workers" and water?
I believe AlphaFold, AlphaEvolve etc are _not_ looking to get to AGI. The whole article is a case against AGI chasing, not ML or LLM overall.
AlphaEvolve is a general system which works in many domains. How is that not a step towards general intelligence?
And it is effectively a loop around LLM.
But my point is that we have evidence that Demis Hassabis knows his shit. Just doubting him on a general vibe is not smart
AlphaEvolve is a system for evolving symbolic computer programs.
Not everything that DeepMind works on (such as AlphaGo, AlphaFold) are directly, or even indirectly, part of a push towards AGI. They seem to genuinely want to accelerate scientific research, and for Hassabis personally this seems to be his primary goal, and might have remained his only goal if Google hadn't merged Google Brain with DeepMind and forced more of a product/profit focus.
DeepMind do appear to be defining, and approaching, "AGI" differently that the rest of the pack who are LLM-scaling true believers, but exactly what their vision is for an AGI architecture, at varying timescales, remains to be seen.
Has he, his team, or DeepMind used any AGI rhetoric, even just as advertising?
Hassabi has talked about AGI in a lot of interviews. So has members of his Deepmind team. And of course current and former alphabet employees - the most prominent being schmidt. He definitely thinks it is coming and said we should prepare for it. Just search for his interviews on AI and you'll get a bunch of them.
This entire thread: people who are too lazy to read basic info about AI companies but have an opinion about "AGI rhetoric".
What do you think OpenAI was founded in a response to?
Musk’s wild-eyed AGI visions and hysteria towards the sober-minded, research-focused efforts of DeepMind / Demis Hassabis?
Yeah, in reality it seems that DeepMind are more the good guys, at least in comparison to the others.
You can argue about whether the pursuit of "AGI" (however you care to define it) is a positive for society, or even whether LLMs are, but the AI companies are all pursuing this, so that doesn't set them apart.
What makes DeepMind different is that they are at least also trying to use AI/ML for things like AlphaFold that are a positive, and Hassabis' appears genuinely passionate about the use of AI/ML to accelerate scientific research.
It seems that some of the other AI companies are now belatedly trying to at least appear to be interested in scientific research, but whether this is just PR posturing or something they will dedicate substantial resources to, and be successful at, remains to be seen. It's hard to see OpenAI, planning to release SexChatGPT, as being sincerely committed to anything other than making themselves a huge pile of money.
Hao is not just a "ai is bad" book... Those exist but Hao is a highly credited journalist.
I’m not sure you understand what AGI is given the citations you’ve provided.
> "While AlphaEvolve is currently being applied across math and computing, its *general* nature means it can be applied to any problem whose solution can be described as an algorithm, and automatically verified. We believe AlphaEvolve could be transformative across many more areas such as material science, drug discovery, sustainability and wider technological and business applications."
Is that not general enough for you? or not intelligent?
Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
> Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
AGI means it can replace basically all human white collar work, alpha evolve can't do that while average humans can. White collar work is mostly done by average humans after all, if average humans can learn that then so should an AGI.
An easier test is that the AGI must be able to beat most computer games without being trained on those games, average humans can beat most computer games without anyone telling them how to do it, they play and learn until they beat it 40 hours later.
AGI was always defined as an AI that could do what typical humans can do, like learn a new domain to become a professional or play and beat most video games etc. If the AI can't study to become a professional then its not as smart or general as an average human, so unless it can replace most professionals its not an AGI because you can train a human of average intelligence to become a professional in most domains.
AlphaEvolve demonstrates that Google can build a system which can be trained to do very challenging intelligent tasks (e.g. research-level math).
Isn't it just an optimization problem from this point? E.g. now training take a lot of hardware and time. If they make it so efficient that training can happen in matter of minutes and cost only few dollars, won't it satisfy your criterion?
I'm not saying AlphaEvolve is "AGI", but it looks odd to deny it's a step towards AGI.
I think most people would agree that AlphaEvolve is not AGI, but any AGI system must be a bit like AlphaEvolve, in the sense that it must be able to iteratively interact with an external system towards some sort of goal stated both abstractly and using some metrics.
I like to think that the fundamental difference between AlphaEvolve and your typical genetic / optimization algorithms is the ability to work with the context of its goal in an abstract manner instead of just the derivatives of the cost function against the inputs, thus being able to tackle problems with mind-boggling dimensionality.
The "context window" seems to be a fundamental blocker preventing LLMs from replacing a white collar worker without some fundamental break through to solve it.
It's to early to declare something "fundamental blocker" while there's so much ongoing research.
There's been 'ongoing research' since the 60s
Isn't the point that DeepMind is producing products providing value to humanity, where AGI looks like something that will produce mainly harm?
How refreshing to see an AI realistic view on HN these days! As the author said, no one is claiming transformer tech useless, the issue is the relentless drive to claim loudly that transformers will lead to AGI in the next few years and solve all existing problems and that it is worth the current negative damage to society and the environment.
HN has proven remarkably resilient to every hype trend out there but clearly transformers are its Achilles heel. That or/and massive transformer astroturfing
I did not get the sentiment here on HN that AGI is right behind the next corner.
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
do you find a 40-60% failure rate fits your definition of correctness? I don't think they really needed to spell this failure out...
https://www.salesforce.com/blog/why-generic-llm-agents-fall-...
They are not “correct” most of the time in my experience. I assumed the author left that “proof” out because it was obvious.
[dead]
> and exploits and traumatises data workers to make sure ChatGPT doesn’t generate outputs like child sexual abuse material and hate speech or encourage users to self-harm.
It's hard to take this seriously, especially with the inflated language.
They're essentially doing moderation. It's a job that's been done and needed on internet platforms of all stripes for at least 30 years. Sure, it can been unpleasant work. If it's "traumatizing" you, you shouldn't be doing it. Acting like this is some novel and horrific phenomenon springing from the quixotic pursuit of AGI is ridiculous. It would be needed even if no one believed AGI was possible.
Greedy managers are a blocker to actual engineering. It wasn't enough that they were trying to squeeze the last ounce of delivery via twisted implementations of Agile. Now they are shooting down every attempt to apply any amount of introspection and thought with blanket expectation of LLMs obviating any need to do so. That combined with random regurgitation of terms like "MCP" and "agentic" has made programming into a zombie-like experience of trying to coax the LLMs to produce something workable while fighting inflated expectations of the hallucinating bosses.
> Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value
I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
> I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
This is the new line now that LLMs are being commoditized, but in the post-Slate Star Codex AI/Tech Accelerationist era of like '20-'23 the Pascal's wager argument was very much a thing. In my experience it's kind of the "true believer" argument, whereas the ROI/productivity thing is the "I'm in it for the bag" argument.
Right. Nobody makes a Pascal's wager-style argument in _favor_ of investing in AGI. People have sometimes made one against building AGI, on existential risk grounds. The OP author is about as confused on this as the water usage point... But the appetite for arguments against AI (which has legitimate motivations!) is so high that people are willing to drop any critical thinking.
People have definitely made the argument that every day AGI is delayed, people are dying from things AGI could have cured etc.
That's not Pascal's Wager unless they're saying AGI has infinitesimal probability but infinite payoff, or something like that. If they think AGI is likely, they may be wrong but it's just technological optimism.
"The argument" ignores the opportunity cost of the other potential uses of the invested resources.
[dead]
After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity. Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
Can you elaborate on that last sentence a bit? How has engineering lost touch?
What I mean is the process of brainstorming with other engineers or understanding a complex solution from a more senior engineer is fading away. There are lots of tools with AI that do it for you. The human experience of connecting is going away.
The act of putting characters in files isn't the value.
I think he got it backwards. Whisper, incredible things chatbots can do with machine translation and controlled text generation, unbelievably useful code-generation capabilities (if enjoyed responsibly), new heights in general and scientific question answering, etc. AI as a set of tools is just great already, and users have access to it at a very low cost because these people passionately believe in weirdly improbable scenarious and their belief is infectious enough for some other people to give them enough money for capex and for yet other people to work 996 if not worse to push their models forward.
To put it another way, there were many talented people and lots of compute already before the AI craze really took off in early 2020s, and tell me, what magical things were they doing instead?
I feel the same frustration, seen from another angle : I think we can use current ML techniques to solve 3 or 4 hard problems in 3D reconstruction, and doing so would unlock a vast amount of value - we could turn lidar scans and photos of buildings and industrial plants into accurate 3D models automatically.
BUT I think the bottleneck is _funding_ of small early risky startups to do the needed engineering work.
My notes on this : https://quantblog.wordpress.com/2025/10/29/digital-twins-the...
LLMs, GPU datacenters attract all the big money, and the med and small VCs seem to be leaving their money in the bank earning high interest rates, unless there is a slam dunk opportunity with guaranteed traction and MRR growth.
We seem to be betting that only the large companies will innovate, when historically this has not been the case - Deepseek is a recent counterexample.
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
I agree with the first two points, but as others have commented the environmental claim here is just not compelling. Starting up your computer is technically creating environmental waste. By his metrics solving technical problems ethically is impossible.
Perfect harmlessness is impossible. Thus, we cannot differentiate between harms, nor should we try. This a stupid thing to profess and I do not believe you would defend it if pressed.
I appreciate your thoughtful reply; I guess I would like more elaboration from the author on what constitutes ethical technology.
As long as you use the most efficient computer (Apple M5 natch) then you're causing minimal waste.
It's probably unethical to solve a problem you could do yourself in 5 minutes at the energy cost of a packet of crisps by using.....lets say....a kilogram of coal. Or two kilos or whatever. But you can't see the cost so it seems free. You might see the money but we haven't linked this to environmental cost yet.
The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
Many arguments I see from people skeptical of AI or other revolutionary theory's seem to overlap.
I'm confident the same arguments could have been used in the industrial revolution....the industry rapidly outgrowing the supply of energy, the overinvesting by people hoping they found a golden goose by being first to market etc.
AGI can't be trained to develop consciousness when we don't even scientifically understand it yet.
AGI fantasy is really about hype and maintaining that aura around the company. It’s to get clout and have people listen. This is what makes company’s valuation shoot up to a trillion dollars.
Why is AGI so hungry? Our brains learn on the fly and do amazing things and how much energy do they use?
So IMO we certainly haven't hit the "right technology" yet and the attempt to achieve it by spending billions, building nuclear power plants etc, is vulnerable to a technical development. So why should we screw up our environmental situation for something that can obviously be done in a vastly better way on the energy equivalent of 3 square meals a day?
Your brain has more capacity than an average LLM, and a self-supervised training regime etched into it by millions of years of evolution. Its inductive bias is very well tuned for the environment. It still takes decades to build itself up to peak performance.
And at its peak? Human brain doesn't actually have an overwhelming advantage over an LLM. It's a mixed bag of advantages and disadvantages.
LLMs think fast, and can input and output data much faster than a human. But they struggle to work on the same task for a long time, and have a problem with visual inputs and object manipulation. LLMs have more knowledge in total, but humans have better meta-knowledge, which is useful for hallucination avoidance. LLMs can only learn in context efficiently, but humans learn continuously and retain what they learned. LLMs and humans are currently trading blows when it comes to inference energy efficiency - especially when you account for things like sleep or rest.
I don't think there's a "right technology" at all. There may not be a state-change upgrade that gets us x100000 on a dime and goes all the way to an AGI on every smartphone and an ASI in any datacenter worth the name. I expect there to be a lot of little +5% and +10% upgrades that add up over time.
The history of AI has been to reject human inspired forms and then be forced to adopt them.
OK that's a sweeping statement but I think it encapsulates the general attitude quite well for a single sentence. We fly in planes which are not like birds but they have wings and tails and flaps and slats and we're even getting into making the wings warp.
Sleep is needed for memory I think so there is a price to pay for continuous learning. It's important to be able to forget too. - to rotate the logfiles so to speak. AGI only interests me because it will be able to understand us - therefor it has to work like us and experience things like us.
I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
Admitting you need to do basic research is admitting you're not actually <5 years from total world domination (so give us money now).
We are yet to see a pure theoretical roadblock between LLMs and AGI. The way things are going, I wouldn't be surprised if an existing LLM architecture (whether fully transformer-based or one of the hybrids) can hit AGI with the right scale, training and some scaffolding.
On the other hand, extracting usable insights from neuroscience? Not at all easy. Human brain does not yield itself to instrumentation.
If an average human had 1.5 Neuralink implants in his skull, and raw neural data was cheap and easy to source? You bet someone would try to use that for AI tech. As is? We're in the "bitter lesson" regime. We can't extract usable insights out of neuroscience fast enough for it to matter much.
Many of the people in control of the capital are gamblers rather than researchers.
If you want to create artificial human intelligence you need to know how the brain works. If you're creating alien intelligence the brain doesn't matter.
Why should they care as long as selling shares of a company selling access to a chatbot is the most profitable move?
I think there’s a jealousy angle to Musk’s need to characterise Hassabis as evil. The guy is actually legitimately smart, and clearly has an endgame (esp medicine and pharmaceuticals) and Musk is just role playing.
I would love to have witnessed them meeting in person, as I assume must have happened at some point when DM was opened to being purchased. I bet Musk made an absolute fool of himself
I believe in AGI because I believe you can estimate where a line is going by its slope. Is there not a way to evaluate the resilience of a rate of change? Like the variance of that rate of change? If so you could almost prove with greater and greater certainty that we will get there.
what looks exponential can be the left side of a sigmoid. true or false?
> But AGI arguments based on EV are nonsensical because the values and probabilities are made up and unfalsifiable.
Hmm, so most businesses behave nonsensically, because they estimate future outcomes…
I’m not disputing the conclusion, but this crucial argument doesn’t seem very strong.
Actual engineering is happening. Every great innovation is fantasy until it's real. What would you rather the money be spent on and why should anyone care? You can call any paid work exploitation if you want to.
Data centers are good because they are resource hogs that are designed by the some of the most technically gifted people in the US. This will hasten gains in water and energy availability
So hysterical levels of investment still comes back to the Kelly Criterion… at the risk of sounding apophenic; the influence Bell Labs continues to have on our world amazes me more every day.
Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White!
https://en.wikipedia.org/wiki/Demis_Hassabis
A game that most certainly did not live up to the hype of its own time, sadly.
It's intellectual charlatanism or incompetence.
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
> And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value).
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Yeah, but their past history should be taken into account here. Altman and musk are just confidence men. what they’ve touched in the past has turned to crap, and it’s only been the people around them that have made anything work despite those people mucking it up.
trust past history as an indicator of future action. In this case, sure some neat stuff will come out of it. But it won’t be nearly what these people say it is. They are huffing each other’s farts.
Do they believe that, or do they need to project confidence in that vision to hit investment targets? I can't tell. Probably a mix.
Money influences thinking so undoubtedly it's a mix, but I think a lot of HNers discount the former, when it plays a very large role. E.g. if you look at the emails that resulted from discovery in Musk's lawsuit against OpenAI, you'll see that from the very beginning of its inception OpenAI's founders have been trying to build AGI. This wasn't a marketing term that was made up years into OpenAI after it blew up and needed to dance in front of investors. This was an explicit goal of OpenAI from the very beginning.
And besides, if they don't reach AGI, they might reach market domance, which is their real goal.
That's still Pascal's Wager but with different (better) probabilities.
Pascal's Wager is based essentially on low probabilities. You can't really say something with different probabilities is Pascal's Wager.
"Probability of X > 90%. Therefore I act as if X will happen." is not Pascal's Wager. That's a bog-standard reiteration of any belief.
What is the definition of Pascal's Wager in your mind?
I always found it funny OpenAI staff tried to delay the release of GPT to the world because they feared the consequences of giving the public such a power. Hearing stuff like this makes it even funnier:
> In the pit, [Sutskever] had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained represented a good, aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. … Sutskever doused the effigy in lighter fluid and lit on fire.
Sutskever was one the people behind the coup of Sam Altman over AI safety concerns. He also said this in 2022:
> "It may be that today's large neural networks are slightly conscious." [1]
A good question is are these AI safety proponents a bit loony or do they actually believe this stuff. Maybe it's both.
[1] https://futurism.com/the-byte/openai-already-sentient
We don't know enough about consciousness to be able to conclusively confirm or deny that LLMs are conscious.
Claiming otherwise is overconfident stupidity. Of which there is no shortage of that in AI space.
That's the sort of convenient framing that lets you get away with hand wavy statements which the public eats up, like calling LLM development 'superintelligence'.
It's good for a conversation starter on Twitter or a pitch deck, but there is real measurable technology they are producing and it's pretty clear what it is and what it isn't.
In 2021 they were already discussing these safety ideas in a grandiose way, when all they had was basic lego building blocks (GPT 1). This isn't just a thought experiment to them.
As they well should have. Because they have more foresight than a pile of rocks.
They knew they had the beginnings of an incredibly capable technology at their hands, and they knew that intelligence is an extremely dangerous thing.
And so far? The capabilities of today's systems are already impressive, and keep improving. If you're thinking "what those systems are doing today isn't that bad", you shouldn't be. Concern yourself with the capabilities of a bleeding edge AI from year 2035.
>…Musk would regularly characterise Hassabis as a supervillain who needed to be stopped. Musk would make unequivocally clear that OpenAI was the good to DeepMind’s evil. … “He literally made a video game where an evil genius tries to create AI to take over the world,” Musk shouted [at an OpenAI off-site], referring to Hassabis’s 2004 title Evil Genius, “and fucking people don’t see it. Fucking people don’t see it! And Larry [Page]? Larry thinks he controls Demis but he’s too busy fucking windsurfing to realize that Demis is gathering the power.”
There are some deeply mentally ill people out there, and given enough influence, their delusions seem to spread like a virus, infecting others and becoming a true mass delusion. Musk is not well, as he has repeatedly shown us. It amazes me that so many other people seem to be susceptible to the delusion, though.
[dead]
To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
https://www.youtube.com/watch?v=A-b7-fLOjlY
But it's a boon to gathering investment capital and talent.
Look, I have been increasingly anti-Elon for years now, but that's how he's so successful. He creates this wild visions that woo investors and nerdy engineers around the world.
That's the whole point. If his pitch was "we can create better chat bots" no one would care.
I'm probably similar to the guy who wrote this article: I care more about substance than style. That doesn't get you ahead in this world, though. Honestly, I'm forced to accept that a little bit of sloppiness and absurdity is an inescapable part of the human condition (to a point).
> I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
I wonder if Tom Phillips isn't very familiar with the history of Silicon Valley, because this is the most unremarkable thing. 60 years ago personal computers were sci-fi fantasy, and 50 years ago they were a mainstream view in Silicon Valley. 50 years ago usable GUIs were sci-fi fantasy, and 40 years ago they were a mainstream view in Silicon Valley. 40 years ago a global computer network used by almost everyone was sci-fi fantasy, and 30 years ago it was a mainstream view in Silicon Valley. 30 years ago always-internet-connected wearable computing devices were sci-fi fantasy, and 20 years ago they were a mainstream view in Silicon Valley. 20 years ago renewable energy cheaper than coal was a sci-fi fantasy, and 10 years ago it was a mainstream view in Silicon Valley.
There were a lot of other sci-fi fantasies that didn't pan out, of course, or at least haven't panned out yet, so this isn't a sign that AGI will definitely happen. Xanadu, the Information Superhighway, graphical programming, virtual reality, freedom, world peace, energy too cheap to meter, the full automation of production, flying cars, and so on. But the entire point of Silicon Valley is that it's a machine for turning sci-fi fantasies into the realities of everyday life.
Keeping up the ruse is the only way to justify the sunk cost in major investment.
AI is also a blocker to art. Going to art school seems kind of pointless these days.
AI is a blocker to the human experience.
Calling it a fantasy is a biased and limited thinking
"Reading Empire of AI by Karen Hao, I was struck by how people associated with OpenAI believe in AGI. They really do think someone, perhaps them, will build AGI, and that it will lead to either the flourishing or destruction of humanity."
"I think it's remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley."
I think the use of the term "believe" is remarkable
According to the "AI experts" there are "believers" and "skeptics"
Science fiction is exactly that: fiction
For decades, software developers cannot stop using the word "magic", "magically", etc.
Silicon Valley is a place for believers
A place where promoters like Steve Jobs can, according to one Apple employee, distort reality^1
1. https://en.wikipedia.org/wiki/Reality_distortion_field
https://en.wikipedia.org/wiki/P._T._Barnum
Some people enjoy science fiction and fantasy. Others may not care for it. It's a matter of taste
>AGI fantasy...
>I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
Human like thinking by machines, which I think is what most people think of as AGI was not until recently a sci-fi fantasy.
It was dealt with by Turning of Turing test fame and the main founder of computer science, around 1950 and the idea of singularity in the tech sense came from John von Neumann who was fundamental to the John von Neumann architecture used as the basis of much computing. If you assume the brain is a biological computer and electronic computers get better in a Moore's law like way then a crossover is kind of inevitable.
Dismissing it as sci-fi fantasy seems a bit like dismissing a round as opposed to flat earth ideas in a similar way.
Which doesn't mean that LLMs are the answer and we should stick all the money into them. That's a different thing.
Where is all the moral outrage that completely stonewalled technologies like human cloning? For what most businesses want out of AGI, it's tantamount to having digital slaves.
Likely because (very?) few would associate LLMs in their current form with "digital slaves". Attributing personhood to a non-corporeal entity is likely a multi-generational change, if it ever happens.
One symptom of AGI fantasy that I particularly hate is the dismissal of applied AI companies as "wrappers" - as if they're not offering any real technical add on top of the models themselves.
This seems to be a problem specific to AI. No one casts startups that build off of blockchains as thin, nor the many companies that were enabled by cloud computing and mobile computing as recklessly endangered by competition from the maintainers of those platforms.
The reality is that applying AI to real challenges is an important and distinct problem space from just building AI models in the first place. And in my view, AI is in dire need of more investment in this space - a recent MIT study found that 95% of AI pilots at major organizations are ending in failure.
It is such a pure thing when an engineer looks at the world and is surprised, frustrated, or disappointed at behavior at scale. This is a finance game which in itself is a storytelling / belief based system. It might seem like math, but when you're playing on the growth edges valuation is really is about the story you tell and the character of the players. Thats only worse when people stop caring about cashflows or only expect them to happen "in the future" because that makes it someone else's problem.
I get the skepticism about the dramedy of burning future AGI in effigy. But given humans are always a dramady, I don’t judge odd or hyperbolic behaviors too harshly from a distance.
It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with.
I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like?
AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier.
This isn’t writing. Although that changed everything.
This isn’t the printing press. Although that changed everything.
This isn’t the telegraph. Although that changed everything.
This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything.
This is intelligence. The meta technology of all technology.
And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere.
The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress.
It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way.
Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”.
Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve.
It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think.
Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity.
Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding.
Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like.
Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us.
What an odd and transparently motivated thought.
While I agree that the current LLM-based approaches won't get us to (sentient) AGI, I think this article is missing a key point: the entire modern AI revolution (while founded on research work esp coming from Google) was fired up by the AGI dreamers at OpenAI with GPT3+ then ChatGPT etc. They were first in industry; they created the field.
Even if you don't expect them to get us over the final line, you should give them credit for that.
The lay misconception and wrongly attributed revolutions, discoveries, inventions is so common it has a name - Stigler’s law of eponymy.
You confound the AI product with the AI revolution.
You confound my point with some other of your own misreading.
To repeat: The claim of the article is clearly incorrect. It's precisely the AGI "fantasists" that have engineered the incredible technological landscape we now live in.
If you disagree, it would be useful to frame your argument in a clearer and less condescending fashion.
Thought this was going to be a good article then the author started mentioning water consumption and I stopped reading.
AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent.
In the meantime, 100% agree, it's complete fantastical nonsense.
Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
sorry to reply again, but it also sounds as if the lack of context is causing a problem. The word weird terms on a certain emotion and tone of voice. If this were in person, the other party might have a tone and demeanor that shows that word "weird" indicates a trailing off, a need for pause and contemplation, not a potential pejorative.
questioning someone in an academic matter further, just revert to the academic literature around psychology and therapy, where someone reflects in a literal way upon what they said. The LLM could easily have responded that it was just a trailing stray comment meant to indicate inquisitiveness rather than deflection. if this were real intelligence, it might take a moment to automatically reflect on why it used the word “weird“ and then let the user know that this might be a point of interest to look into?
it sounds like they are trained to be a confidence man executive. hype things and blow smoke. It's able to form a response when questioned carefully about the patterns created; that’s the only plus I am seeing from your point of view on this particular use of the technology.
[dead]
uh, yeah no shit
[dead]
[dead]
[flagged]
"As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)."
As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
Okay, so come up with an alternative, it's math, you can also write algorithms.
I can’t test them, though.
What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy:
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
The human brain existing is proof that "Gödel-style incompleteness" and "Wolfram's principle" are not barriers to AGI.