This is one of the most artfully positioned, full-throated, thoughtful, and capable takedowns of the AI hype (not necessarily LLMs as tools) I've read. It doesn't try to speak doom to AI companies, it doesn't suggest the tools are worthless at things they're good at. Instead, it disassembles the assumptions underlying ideas about frontier AI labs' promises (of both utopia and doom) by explaining in simple terms what the real accomplishment they've made is:
They are thieves who turned their stolen goods into a pretty neat invention.
They are not prophets of the end times or messianic figures bringing in a new age of human relaxation or the biggest trillionaires since Tim Cook and Elon Musk. They are people with one good tool that they think can and should be used to do everything. They won't accept that their tool is just useful, or helpful. They want it to be _more important than the humans and human work it was based on_, and nothing less will do. They want to replace effort and thought with simulacra of both and be applauded for it.
Lest this go down as an anti-AI post: I use AI tools. I use LLMs every day. They are useful... but they can't do on their own what we did not demonstrate how to do first, because they are tools.
I could be wrong, but I don't know that I see this changing.
> but they can't do on their own what we did not demonstrate how to do first, because they are tools.
Funny thing to say here, where an agent swarm (1M hours agent time) just did something no human did. If someone else solved NS blowup before OpenAI we would have heard their claims already. And we did hear partial claims, "they rushed in the same direction" kind of claims, not "they copied my fully developed idea". I think this is Move 37 moment for math, when we realize we might not be the top anymore.
That is very much the case at the current iteration of AI, but that should change with the introduction of world-mapping, computer vision, and world interacting AI. If machines are built out so that they are constructing accurate world-maps of how the physical world works and how to operate in it, then AI will begin to be able to out-invent human tool concepts and push past the horizon of what humans have so far been able to dream
I think that's the wrong angle with machine learning.
The biggest limits on machine learning are how many examples it takes to get good at something. Right now, this is at a literally evolutionary pace: our bodies/genomes have barely adapted to the invention of controlled fire, thousands of generations ago, yet if it took thousands of examples for our brains to learn then we'd sill be unable to control fire.
Computers only partially make up for being slow in the "by number of examples needed" sense by being fast in the "wall clock" sense: If a biological brain took as many examples to learn a thing as machine learning takes, the creature would starve before figuring out how to eat; yet transistors operate faster than synapses by the same magnitude to which a jogger outpaces continental drift.
The "only partially" is why, despite around a trillion miles of driving experience across the fleet, you still can't buy a Tesla without a steering wheel (and even the Cybercab that's supposed to be without a steering wheel is often photographed with one).
This is a very basic misunderstanding of machine learning. The goal is always to learn a representation that generalizes beyond the training set without overfitting. If you succeed in generalizing, you can do more than produce derivative work.
> or input generated by them as derivatives of human input
Trained mostly on their own data, not ours. The mix changed, from 10% to 90% synthetic data. RL training is now the largest part of the process. The AI work is searching, and I don't think we can claim this process to be credited to us anymore. The origin point is, but not the journey.
That’s the thing. AI won’t be limited to being trained on only human input, why would it? Humans are trained by interacting with their environment plus a genetic world-mapping initial seed. There is no reason AI will not be the same, just at a much more extreme level
It depends on the knowledge domain. I would argue that much of human knowledge is individual to the person acquired by interacting with their environment
> History judges researchers who work with the bad guys ... To them I say: learn from history and get out now. You’ll still be ahead of the curve. The current backlash against AI may seem huge, but it’s nothing compared to what I think the backlash will be in coming years.
> I chose 7 years because I see certain parallels between the situation with researchers and Big AI today and the relationship between researchers and Epstein in 2019.
Look, I absolutely love Vi Hart's stuff. And I mean love love, and I have for almost 20 years.
But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I thought the original Mathathon sounded like a great idea. They shut that down, and I think the revised one seems like a half decent idea. But if I get involved in it, in 7 years it'll be the same as if I had been involved with Jeffrey Epstein?
I feel like the discourse on this is starting to get way out of control. I say that with no less love for Vi Hart's ideas than I have always had. But surely this technology can be used for some good?
Some of the smartest people I know are also some of the people that use AI the most.
They use their time and energy on solving hard problems. Either in the abstract (research) or more practical problems (entrepreneurship).
I completely fail to see how anyone would want to equal researchers and entrepreneurs to criminals.
(a) AI will replace all or nearly all human intellectual labour
(b) AI will achieve superintelligence and become a (hopefully benevolent) digital god
(c) AI will be a superweapon that defines the next era of warfare
(d) AI will render art and creativity obsolete
These are real arguments I have heard. I'm sure you have too. If any of them are true, even partly, would you really be shocked to see the people of the future looking back at us as a bunch of Chamberlains?
And would they judge us less harshly just because the people who wanted to end the economy, summon Cthulhu, build a nuke and kill art simply failed to achieve their goals? Is the goal itself not odious enough to condemn?
People are still arguing about capitalism vs communism, even though the laissez faire capitalism that inspired literal communism ended with the New Deal and literal communism fell some time around the Berlin Wall (give or take when China became 'state capitalism' in practice if not name).
One thing is, I am confident, going to be true regardless of AI: for as long as humans are still around, we'll be arguing about the economy and what 'good' even means in that context.
Less of a quibble:
> (b) AI will achieve superintelligence and become a (hopefully benevolent) digital god
While supremely hubristic, is that really odious given the 'hopefully benevolent'?
Like, I don't believe in any god, but is this any more odious than someone simply praying for success to a god they do believe in?
> But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I'm not saying he did it (how would I tell?), but Altman has absolutely been accused by his own sister.
And while Musk doesn't seem to have actually gone to that island, he did seem interested.
And one of the files did show Zuckerberg and Musk dining with Epstein.
And many of the big tech leadership did openly support Trump, despite the long list of things that now includes suspicious gaps in which of the Epstein files actually got published. (Even if theres a legit reason behind what remains hidden, promising to publish then failing is at best still a huge own-goal because "they have something to hide" is necessarily true and always sounds like it means "that they themselves did").
So, yeah, I can see that for at several of the big names.
(There's also a lot of distaste for Anthropic; I'm not aware of anything of this category, but even without that it is easy to tar everyone with the same brush).
Any expensive search in an exponential space meets cost very soon, but Vi is a Platonist, she thinks AI generate meaningless tokens, such as enumerating all proofs by LEAN. No, that search is interacting with its own costs, it is not platonic search scaled to infinity.
> Imagine asking an AI to generate the whole numbers in order, from 0 on up. You could spend trillions of dollars having it fill up datacenter after datacenter with lists of numbers. The more we spend on the project, the more the list would start including some very big and important numbers. But what would actually be accomplished?
> Consider this and then consider how it applies to every number in any dataset and everything an AI generates: with our normal human understanding of a number, the number theoretically contains all of the meaning of that number. A number intrinsically contains the fact of whether it's prime, whether it's irrational, whether it's a counterexample to the Riemann Hypothesis, all facts known and unknown. You don't need a human to interpret the number in order for the number to mean what it means. In fact, the longer a human spends interpreting, the more meaning they might find that a number intrinsically has in and of itself. And yet... can you think of a less meaningful way to fill up a datacenter than with an ordered list of numbers?
> Listing all possible proofs out of context, whether in order or randomly, would hardly be more meaningful than listing numbers. Almost all possible proofs are trivial and meaningless, mathematicians select for the fun stuff.
> But I will say, since it seems every player in the US government and economy have committed to all jumping off a cliff together in a stubborn wish to cover the country in datacenters, I could think of worse applications than to fill them up with every possible mathematical proof below a certain size.
> You'd end up with a Library of Babel of proofs.
> You can imagine the quality of literature you'd get if you trained an LLM on the Library of Babel, which contains every possible book (or combination of characters that fit in 410 pages). The library, by itself, functionally contains no information whatsoever. The LLM trained on it would be a literal smooth-brain with no correlations to wrinkle up those gradients.
> Without the context of language and how it’s used by people, a great work of literature is just another possible random combination of letters. This can be represented as a point in multidimensional space, and a computer can guess that maybe there's more things of interest to humans near where the stuff we already like is, but only humans know how to push further into unknown directions of multidimensional space that are interesting and worthwhile.
> Whenever a company or person shares something “AI generated” that seems pretty good, that tells me that the AI must necessarily be using a lot of human-created data in that same area. The AI can not have leapt there on its own, even if it might look like it did to a person not aware of the scope of human-made work that went into it.
> The Mathlib of Babel problem is one reason why, in my view, formal math and AI are just one tool that might be useful for some math sometimes, not the end of mathematics or the future of mathematics. Formal mathematics and algorithmically-assisted proofs are not my personal aesthetic but I think it should be somebody's, and it's not like our fragmented field doesn't know how to handle another specific expertise joining the mix. We've already folded them in.
Found this interesting and thought-provoking. Why do we care about these proofs? Why do we care about this art, or this code? Because it is what is useful and interesting to humans.
I'll take this over AI writing any day. Far easier to read. Yeah it could probably do with another edit pass, but it's night and day easier to read than the slop I see from most blogs now. AI is great for many things, but writing engaging long-form content is far from one of them.
This is one of the most artfully positioned, full-throated, thoughtful, and capable takedowns of the AI hype (not necessarily LLMs as tools) I've read. It doesn't try to speak doom to AI companies, it doesn't suggest the tools are worthless at things they're good at. Instead, it disassembles the assumptions underlying ideas about frontier AI labs' promises (of both utopia and doom) by explaining in simple terms what the real accomplishment they've made is:
They are thieves who turned their stolen goods into a pretty neat invention.
They are not prophets of the end times or messianic figures bringing in a new age of human relaxation or the biggest trillionaires since Tim Cook and Elon Musk. They are people with one good tool that they think can and should be used to do everything. They won't accept that their tool is just useful, or helpful. They want it to be _more important than the humans and human work it was based on_, and nothing less will do. They want to replace effort and thought with simulacra of both and be applauded for it.
Lest this go down as an anti-AI post: I use AI tools. I use LLMs every day. They are useful... but they can't do on their own what we did not demonstrate how to do first, because they are tools.
I could be wrong, but I don't know that I see this changing.
> but they can't do on their own what we did not demonstrate how to do first, because they are tools.
Funny thing to say here, where an agent swarm (1M hours agent time) just did something no human did. If someone else solved NS blowup before OpenAI we would have heard their claims already. And we did hear partial claims, "they rushed in the same direction" kind of claims, not "they copied my fully developed idea". I think this is Move 37 moment for math, when we realize we might not be the top anymore.
Did you read the article? It goes into a lot of detail about how oversold the Navier-Stokes discovery is.
That is very much the case at the current iteration of AI, but that should change with the introduction of world-mapping, computer vision, and world interacting AI. If machines are built out so that they are constructing accurate world-maps of how the physical world works and how to operate in it, then AI will begin to be able to out-invent human tool concepts and push past the horizon of what humans have so far been able to dream
I think that's the wrong angle with machine learning.
The biggest limits on machine learning are how many examples it takes to get good at something. Right now, this is at a literally evolutionary pace: our bodies/genomes have barely adapted to the invention of controlled fire, thousands of generations ago, yet if it took thousands of examples for our brains to learn then we'd sill be unable to control fire.
Computers only partially make up for being slow in the "by number of examples needed" sense by being fast in the "wall clock" sense: If a biological brain took as many examples to learn a thing as machine learning takes, the creature would starve before figuring out how to eat; yet transistors operate faster than synapses by the same magnitude to which a jogger outpaces continental drift.
The "only partially" is why, despite around a trillion miles of driving experience across the fleet, you still can't buy a Tesla without a steering wheel (and even the Cybercab that's supposed to be without a steering wheel is often photographed with one).
Why? They're still only going to be trained on human input or input generated by them as derivatives of human input.
Modality change might expand their usefulness, but I don't think it changes the basic parameters of their operation.
Learning to swim might let you see a coral reef, but it won't turn you into one.
This is a very basic misunderstanding of machine learning. The goal is always to learn a representation that generalizes beyond the training set without overfitting. If you succeed in generalizing, you can do more than produce derivative work.
> or input generated by them as derivatives of human input
Trained mostly on their own data, not ours. The mix changed, from 10% to 90% synthetic data. RL training is now the largest part of the process. The AI work is searching, and I don't think we can claim this process to be credited to us anymore. The origin point is, but not the journey.
That’s the thing. AI won’t be limited to being trained on only human input, why would it? Humans are trained by interacting with their environment plus a genetic world-mapping initial seed. There is no reason AI will not be the same, just at a much more extreme level
Aren’t humans also mostly trained on human input?
It depends on the knowledge domain. I would argue that much of human knowledge is individual to the person acquired by interacting with their environment
And much of it inaccessible to others, let alone to AI
> History judges researchers who work with the bad guys ... To them I say: learn from history and get out now. You’ll still be ahead of the curve. The current backlash against AI may seem huge, but it’s nothing compared to what I think the backlash will be in coming years.
> I chose 7 years because I see certain parallels between the situation with researchers and Big AI today and the relationship between researchers and Epstein in 2019.
Look, I absolutely love Vi Hart's stuff. And I mean love love, and I have for almost 20 years.
But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I thought the original Mathathon sounded like a great idea. They shut that down, and I think the revised one seems like a half decent idea. But if I get involved in it, in 7 years it'll be the same as if I had been involved with Jeffrey Epstein?
I feel like the discourse on this is starting to get way out of control. I say that with no less love for Vi Hart's ideas than I have always had. But surely this technology can be used for some good?
Some of the smartest people I know are also some of the people that use AI the most. They use their time and energy on solving hard problems. Either in the abstract (research) or more practical problems (entrepreneurship). I completely fail to see how anyone would want to equal researchers and entrepreneurs to criminals.
Is it still ridiculous in the context of:
(a) AI will replace all or nearly all human intellectual labour
(b) AI will achieve superintelligence and become a (hopefully benevolent) digital god
(c) AI will be a superweapon that defines the next era of warfare
(d) AI will render art and creativity obsolete
These are real arguments I have heard. I'm sure you have too. If any of them are true, even partly, would you really be shocked to see the people of the future looking back at us as a bunch of Chamberlains?
And would they judge us less harshly just because the people who wanted to end the economy, summon Cthulhu, build a nuke and kill art simply failed to achieve their goals? Is the goal itself not odious enough to condemn?
The overall point is valid, but one quibble:
> the people who wanted to end the economy
People are still arguing about capitalism vs communism, even though the laissez faire capitalism that inspired literal communism ended with the New Deal and literal communism fell some time around the Berlin Wall (give or take when China became 'state capitalism' in practice if not name).
One thing is, I am confident, going to be true regardless of AI: for as long as humans are still around, we'll be arguing about the economy and what 'good' even means in that context.
Less of a quibble:
> (b) AI will achieve superintelligence and become a (hopefully benevolent) digital god
While supremely hubristic, is that really odious given the 'hopefully benevolent'?
Like, I don't believe in any god, but is this any more odious than someone simply praying for success to a god they do believe in?
> But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I'm not saying he did it (how would I tell?), but Altman has absolutely been accused by his own sister.
And while Musk doesn't seem to have actually gone to that island, he did seem interested.
And one of the files did show Zuckerberg and Musk dining with Epstein.
And many of the big tech leadership did openly support Trump, despite the long list of things that now includes suspicious gaps in which of the Epstein files actually got published. (Even if theres a legit reason behind what remains hidden, promising to publish then failing is at best still a huge own-goal because "they have something to hide" is necessarily true and always sounds like it means "that they themselves did").
So, yeah, I can see that for at several of the big names.
(There's also a lot of distaste for Anthropic; I'm not aware of anything of this category, but even without that it is easy to tar everyone with the same brush).
To be more precise, she's comparing it to taking money from E*****n despite the vibes being just vibes (_before_ the 2008 conviction)
But yes, MIT Medialab also took money, after the conviction, thus muddying her argument
The more intelligent someone is, can just lead to more creative ways to go crazy.
Time will tell whether it’s a crazy position or not. The best we can do is make our own choices.
It is a crazy position. Time will tell whether it'll get infectious and blow up.
Any expensive search in an exponential space meets cost very soon, but Vi is a Platonist, she thinks AI generate meaningless tokens, such as enumerating all proofs by LEAN. No, that search is interacting with its own costs, it is not platonic search scaled to infinity.
> Imagine asking an AI to generate the whole numbers in order, from 0 on up. You could spend trillions of dollars having it fill up datacenter after datacenter with lists of numbers. The more we spend on the project, the more the list would start including some very big and important numbers. But what would actually be accomplished?
> Consider this and then consider how it applies to every number in any dataset and everything an AI generates: with our normal human understanding of a number, the number theoretically contains all of the meaning of that number. A number intrinsically contains the fact of whether it's prime, whether it's irrational, whether it's a counterexample to the Riemann Hypothesis, all facts known and unknown. You don't need a human to interpret the number in order for the number to mean what it means. In fact, the longer a human spends interpreting, the more meaning they might find that a number intrinsically has in and of itself. And yet... can you think of a less meaningful way to fill up a datacenter than with an ordered list of numbers?
> Listing all possible proofs out of context, whether in order or randomly, would hardly be more meaningful than listing numbers. Almost all possible proofs are trivial and meaningless, mathematicians select for the fun stuff.
> But I will say, since it seems every player in the US government and economy have committed to all jumping off a cliff together in a stubborn wish to cover the country in datacenters, I could think of worse applications than to fill them up with every possible mathematical proof below a certain size.
> You'd end up with a Library of Babel of proofs.
> You can imagine the quality of literature you'd get if you trained an LLM on the Library of Babel, which contains every possible book (or combination of characters that fit in 410 pages). The library, by itself, functionally contains no information whatsoever. The LLM trained on it would be a literal smooth-brain with no correlations to wrinkle up those gradients.
> Without the context of language and how it’s used by people, a great work of literature is just another possible random combination of letters. This can be represented as a point in multidimensional space, and a computer can guess that maybe there's more things of interest to humans near where the stuff we already like is, but only humans know how to push further into unknown directions of multidimensional space that are interesting and worthwhile.
> Whenever a company or person shares something “AI generated” that seems pretty good, that tells me that the AI must necessarily be using a lot of human-created data in that same area. The AI can not have leapt there on its own, even if it might look like it did to a person not aware of the scope of human-made work that went into it.
> The Mathlib of Babel problem is one reason why, in my view, formal math and AI are just one tool that might be useful for some math sometimes, not the end of mathematics or the future of mathematics. Formal mathematics and algorithmically-assisted proofs are not my personal aesthetic but I think it should be somebody's, and it's not like our fragmented field doesn't know how to handle another specific expertise joining the mix. We've already folded them in.
Found this interesting and thought-provoking. Why do we care about these proofs? Why do we care about this art, or this code? Because it is what is useful and interesting to humans.
[flagged]
Vi Hart's claim to fame is extremely verbose and meandering math explanations... hope she keeps doing that, because it's awesome
Glad that this is a real person.
I'll take this over AI writing any day. Far easier to read. Yeah it could probably do with another edit pass, but it's night and day easier to read than the slop I see from most blogs now. AI is great for many things, but writing engaging long-form content is far from one of them.