I just don't understand why people are incapable of extrapolating. AI can't do this blah blah, my brother in spaghetti monster, have you seen the curve? ChatGPT launched in November, 2022. Opus in December last year. Do you see where this is going?
AI may be a tool and add 50 IQ points to a 100 IQ person today but what happens when its adding 10000 IQ points. The 100 base doesn't quite matter as much does it? What does that world look like?
No, it's not just next word prediction. No you can't just wave your hand that it's probabilistic, so insert bad analogy to compiler or calculator and go about your day. And no, you can't just look at its limitations today, make a conclusion, and go back under your rock.
but you’re just on the other end doing the same thing.
one end is luddites nothing to see here, and the other end is omg humanity is cooked because programming is solved! and consciousness is coding so -> gg
yes there’s a wide surface of emergent behavior we haven’t fully explored, but why do you get the pass to draw a straight line to a conclusion? the naysayers are doing the same thing.
At least I have trends backing up my claim. What precedent is there for saying, "Yeah, AI has kept getting better, and keeps doing things that we thought it would never be able to do, but now I think it’s going to plateau because I just can’t see the model writing good poetry or being smart enough to make determinsitic tool calls" or whatever?
No, you don't have trends backing your claims that we are going to add 10,000 IQ points to the average person. Anymore than the S&P 500 is going to 100x in a decade years.
Extrapolation leaves you in a world with no agency. Okay, so the models become superintelligent, you're kind of fucked at that point and there's no value you as a human can add so that reality isn't really productive to think about.
The only thing we can do is look at the current state of things, and we still have the same problems we've always had with these models, they need proper direction and input.
I was earlier of the belief, Oh my god, now nothing is impossible. Ai can market, ai can create video, ai can sell, i can build and make a great product. we will win.
I realized all this was quite wrong, AI just helps people who already know become more efficient. It just gives confidence to people who dont know anything by the sycophantic nature of AI. AI is just a tool for smart people to become smarter.
Love AI and what it allows us to do. But, it does give some superpowers, but the problems that existed before still do and they need to be solved for. with or without AI.
The problem is not AI. It's an excellent technology. The problem is there's an underlying power grab (e.g. layoffs). When humans do that to each other they inherently dehumanize/invalidate/insult each other. Implement strong labor protections or basic income and a lot of this dog-eat-dog toxicity goes away.
Because most automations never capture the complete scope of the job/task ( not even close ). Just like neurons, if you don't use it you lose it and when the inevitable problems come, nobody knows the why, the how and the what. At that point someone smart would incorporate all those real costs and opportunity loses on the "automating everything" equation. But they usually don't.
Of course automating tasks is a must, but it's very far from being a black and white situation. These dynamics have been happening for centuries by now, nothing new.
I agree with that, and that’s why it’s better to ask an AI and improve your prompt, instead of hiring a human that will disappear and you will loose all institutional knowledge
Yes when the sole purpose is to remove the livelihood of people in an attempt to make short term gains for selfish short term reasons....AI is a weapon when you have corporate stooge lackies who have nothing but animous towards fellow humans who dare to work...and try to live. If AI was beneficial, no one would profitize it....but lbr...you know that...
If AI is not beneficial, then those humans could be re-hired soon.
Big companies are always doing bad things, but not because they use AI, because they have legal protections which prevents small companies (which could be anyone like you and me) to compete. The same small companies who could also benefit from using AI.
Depends on the scale and the technology. If the answer is "all of white collar workers" then I would argue it's very toxic. (I don't think it can do that, but it's hard not to get the impression that it's absolutely the goal). I haven't really heard of a stable society with 50% unemployment and zero social safety net.
I don't think anyone ever really believed it would replace ordinary jobs. That angle was meant to appeal to emotion and distract the public from the shady deals and big defense spending.
Think about all the security clearances no longer required to aggregate big data into intelligence reports. The conditions and incentives for LLMs seem almost laser focused on replacing those particular jobs.
It wasn't but ~20 years ago that people were concerned about Google slurping up all the world's data into spying programs. Now that the hardest part to hide is happening, people have forgotten or assumed it already had. Many other smaller and far less capable businesses have come and gone and taken tiny bits of blame until the public was satisfied they knew who the "real" scapegoats were. What they really had were overcomplicated theories built on a nebulous cloud of debatable evidence that led nowhere. This is how it succeeds in plain sight every time.
Clearly not what is being said. If you read dehumanizing your workers and equate it with automating a job, then you're already well into the feeling that humans are fungible pawns to be disposed of, no?
Instead, you can and probably should see technology as augmenting you and your coworkers.
I don’t think humans workers should be cogs in the machine, but from my experience unfortunately that’s how people want to be treated. One simple explanation is that there is no freedom / creativity without responsibility, and the latter is extremely expensive in brain resources
It did replace draftsmen and designers, and changed the work that the engineers do. It's now more efficient to let engineers be their own draftsmen and designers. And design work has changed -- a lot less "hard" quantitative engineering, and a lot more "bureaucracy in three dimensions" as things have gotten more complex.
Normies just don’t understand what all is involved in running and evolving a production system. They think all programmers do is write code. Of course they believe that AI could replace programmers.
My in laws smugly asked me what I would be doing for work since AI would take away all the programming jobs and I said its not gonna happen and they laughed in derision.
I would be glad they don't understand what's next. For now, it helps devs keep a low profile as their work transitions into more bureaucracy. You're lucky they do not yet consider you one of the "bad guys" like lawyers, doctors, teachers, etc. (who didn't get replaced by the photocopier, google, or video tape)
Some people who "used to love programming" have jumped ship for precisely this reason. Ordinary software (not R&D) is getting serious. A broken or disagreeable app is no longer a mere inconvenience. There is growing alignment between business and politics and the older generation is retiring out.
When I think about the typical team composition needed to build a typical app in a lot of companies, well, actually what do I think about? I think, okay, that’s how many people we need to build a typical app - some number. That’s what we should all think … like, what is that “number” of people we need?
So, today, if I think really really really, and I mean, really fucking hard about that “number”, I really really really come up with a number close to 1 (we need one person to type in the instruction at least, maybe two? Maybe three? Definitely not a team … no not that at all).
That means, for everything that we build today, before we even build it, we’ve already mentally replaced people. People be replaced yo, in your mind, in your company, in your reality. That’s it.
It’s a replacement, not a tool. And if you reaaallly think a bit more, you’ll see that it is truly the stupidest possible thing if in your scoping of what needs to be built, you didn’t scope out people. You’d have to be crazy not to scope out people.
Trust me, I never thought we’d get to this point either. I thought it was going to be an “assistant” too.
—-
The layoff numbers still look “normal “ to us. 75k here, 20k there, all numbers we’ve heard before. It hasn’t even really happened yet, the true numbers should be in millions. I didn’t believe the number “trillion dollar company” until, BAM, here’s a trillion dollar company.
Best we can do is kinda get used to the numbers, like the number 0 on one end (how many people you need), and some number N (in millions) that we won’t need anymore. We’ll just slowly get used to these new numbers.
I think if you use AI to automate talking to customers, you're going to stop having customers. If you use AI to do your engineering and deployments, it's up to you to fix it when it goes wrong. If you use it for accounting, you BETTER look at the results if you don't want to be audited. If the AI handles legal, same thing. Basically, if it replaces N people, then you, the one person, need to be accountable for being able to handle N people's roles, because the AI can't take accountability.
Right, the one person (Neo?), has to be able to verify the output. Most of us can’t replace the accountant because we can’t really … replace the accountant. We can replace the accountant’s work, but we can’t replace his/her other work which is knowing if the accounting is right. However, an accountant can replace N other accountants.
One programmer can know if the program is right. So N additional programmers, in that case, can truly be replaced.
I can know the SQL query came back right, I can know the drop down menu looks right, you know what I mean? I can even know if it’s any good. [Person .., ] of Type Y (let’s say Y is “programmer” for now) can be flattened down to 1 in that case (we no longer need the full array of people of a certain type anymore, just the one). Over time we’ll see all different “types” get collapsed down.
> AI can write binary and probably better in a few years.
Assuming you mean an LLM, you'd have to train this LLM entirely on a parameter space of binary tokens. Or are you saying the LLM generating natural language tokens is going to be printing machine language in 3-5 years, because that claim kinda betrays a misunderstanding of LLM functions.
You don't use an LLM to create images, that requires (today) transformer model.
I assume that "AI can write binary" means "AI can use a toolset that results in a binary" because we've already seen GPTs use a combination of LLM and specialized math tools to do the things the original GPTs did.
Right. There is something interesting here that could be explored thoughtfully. The hard part is that we rely on compilers being correct, and they mostly are.
We have no viable mechanism yet to get the same level of confidence if some LLM-based system writes the binary.
Perhaps we can get to a system that produces not just the binary but also a machine-verifiable proof that the binary implements some higher-level language description of the program.
Though then the question will be whether we've gained anything, or whether we've just replaced the compiler with something massively more expensive that does the same thing.
There's some potential here for the LLM-based system to drive better performance optimizations than a regular compiler could.
Of course this isn't what Elon is actually saying, and we'd be better off if fewer people listened to him.
> Perhaps we can get to a system that produces not just the binary but also a machine-verifiable proof that the binary implements some higher-level language description of the program.
We don't even have a solution to the halting problem, and it probably can't be solved. "Proof it implements a spec" is pure science fiction.
Hard agree that we'd all be better off muting Elon Musk though.
I just don't understand why people are incapable of extrapolating. AI can't do this blah blah, my brother in spaghetti monster, have you seen the curve? ChatGPT launched in November, 2022. Opus in December last year. Do you see where this is going?
AI may be a tool and add 50 IQ points to a 100 IQ person today but what happens when its adding 10000 IQ points. The 100 base doesn't quite matter as much does it? What does that world look like?
No, it's not just next word prediction. No you can't just wave your hand that it's probabilistic, so insert bad analogy to compiler or calculator and go about your day. And no, you can't just look at its limitations today, make a conclusion, and go back under your rock.
I’m not convinced model productivity will scale forever.
Presumably there are a lot of use cases that are essentially “solved” where more powerful models won’t matter.
but you’re just on the other end doing the same thing.
one end is luddites nothing to see here, and the other end is omg humanity is cooked because programming is solved! and consciousness is coding so -> gg
yes there’s a wide surface of emergent behavior we haven’t fully explored, but why do you get the pass to draw a straight line to a conclusion? the naysayers are doing the same thing.
At least I have trends backing up my claim. What precedent is there for saying, "Yeah, AI has kept getting better, and keeps doing things that we thought it would never be able to do, but now I think it’s going to plateau because I just can’t see the model writing good poetry or being smart enough to make determinsitic tool calls" or whatever?
it doesn’t have to be that deep or specific.
all things decay on a curve. pretty much all things in life are not modeled by a straight line.
so you’re saying no theres still more UP. sure, but that’s a tale as old as time. when where and how much in how long is the reality piece.
> At least I have trends backing up my claim
No, you don't have trends backing your claims that we are going to add 10,000 IQ points to the average person. Anymore than the S&P 500 is going to 100x in a decade years.
“ChatGPT launched in November, 2022. Opus in December last year.”
You seem to be implying that there was nothing before. Obviously not true.
“AI may be a tool and add 50 IQ points to a 100 IQ person today but what happens when its adding 10000 IQ points.”
What is even IQ? What if it adds 50 now, 25 next month, 12.5 the month after, etc. You should check out Zeno.
Extrapolation leaves you in a world with no agency. Okay, so the models become superintelligent, you're kind of fucked at that point and there's no value you as a human can add so that reality isn't really productive to think about.
The only thing we can do is look at the current state of things, and we still have the same problems we've always had with these models, they need proper direction and input.
That is the question, and it does f** with me. But the existential nihilism that comes with it is not an argument against it not happening.
I was earlier of the belief, Oh my god, now nothing is impossible. Ai can market, ai can create video, ai can sell, i can build and make a great product. we will win.
I realized all this was quite wrong, AI just helps people who already know become more efficient. It just gives confidence to people who dont know anything by the sycophantic nature of AI. AI is just a tool for smart people to become smarter.
Love AI and what it allows us to do. But, it does give some superpowers, but the problems that existed before still do and they need to be solved for. with or without AI.
The problem is not AI. It's an excellent technology. The problem is there's an underlying power grab (e.g. layoffs). When humans do that to each other they inherently dehumanize/invalidate/insult each other. Implement strong labor protections or basic income and a lot of this dog-eat-dog toxicity goes away.
Automating a job is toxic now ?
It can be.
Because most automations never capture the complete scope of the job/task ( not even close ). Just like neurons, if you don't use it you lose it and when the inevitable problems come, nobody knows the why, the how and the what. At that point someone smart would incorporate all those real costs and opportunity loses on the "automating everything" equation. But they usually don't.
Of course automating tasks is a must, but it's very far from being a black and white situation. These dynamics have been happening for centuries by now, nothing new.
I agree with that, and that’s why it’s better to ask an AI and improve your prompt, instead of hiring a human that will disappear and you will loose all institutional knowledge
Yes when the sole purpose is to remove the livelihood of people in an attempt to make short term gains for selfish short term reasons....AI is a weapon when you have corporate stooge lackies who have nothing but animous towards fellow humans who dare to work...and try to live. If AI was beneficial, no one would profitize it....but lbr...you know that...
If AI is not beneficial, then those humans could be re-hired soon.
Big companies are always doing bad things, but not because they use AI, because they have legal protections which prevents small companies (which could be anyone like you and me) to compete. The same small companies who could also benefit from using AI.
Beneficial for who?
> If AI was beneficial, no one would profitize it
What? How does that follow?
Depends on the scale and the technology. If the answer is "all of white collar workers" then I would argue it's very toxic. (I don't think it can do that, but it's hard not to get the impression that it's absolutely the goal). I haven't really heard of a stable society with 50% unemployment and zero social safety net.
200 years ago 75% of the population in western Europe were farmers. 100 years ago 41% were farmers. 10 years ago 1-3% were farmers.
We are looking forward to bring the same productivity gains to logistics and manufacturing (look at the advancements done in the last few decades!).
Why not bring this to white collar work too? I get so much more shit done today than I did a few years ago. It's a great time to be alive!
Has your salary gone up to match your productivity, or are you giving it all away for free?
You're not one of the people who gets to benefit from the gains.
I'm bootstrapping my own company, and I live off savings, so the answer is no. I don't have a salary at all.
I don't think anyone ever really believed it would replace ordinary jobs. That angle was meant to appeal to emotion and distract the public from the shady deals and big defense spending.
Think about all the security clearances no longer required to aggregate big data into intelligence reports. The conditions and incentives for LLMs seem almost laser focused on replacing those particular jobs.
It wasn't but ~20 years ago that people were concerned about Google slurping up all the world's data into spying programs. Now that the hardest part to hide is happening, people have forgotten or assumed it already had. Many other smaller and far less capable businesses have come and gone and taken tiny bits of blame until the public was satisfied they knew who the "real" scapegoats were. What they really had were overcomplicated theories built on a nebulous cloud of debatable evidence that led nowhere. This is how it succeeds in plain sight every time.
Clearly not what is being said. If you read dehumanizing your workers and equate it with automating a job, then you're already well into the feeling that humans are fungible pawns to be disposed of, no?
Instead, you can and probably should see technology as augmenting you and your coworkers.
I don’t think humans workers should be cogs in the machine, but from my experience unfortunately that’s how people want to be treated. One simple explanation is that there is no freedom / creativity without responsibility, and the latter is extremely expensive in brain resources
Automating _my_ job is toxic. Automating other people’s and being employed as a software developer is wholesome chungus moment.
Do you actually talk like this?
When did killing someone else's livelihood stop being sociopathic?
Since the luddites at least
"Computer" used to be a job title for a human. Was it sociopathic to introduce mechanical calculators because it made those jobs unnecessary?
>>> CAD systems did not replace engineers.
It did replace draftsmen and designers, and changed the work that the engineers do. It's now more efficient to let engineers be their own draftsmen and designers. And design work has changed -- a lot less "hard" quantitative engineering, and a lot more "bureaucracy in three dimensions" as things have gotten more complex.
Normies just don’t understand what all is involved in running and evolving a production system. They think all programmers do is write code. Of course they believe that AI could replace programmers.
My in laws smugly asked me what I would be doing for work since AI would take away all the programming jobs and I said its not gonna happen and they laughed in derision.
I would be glad they don't understand what's next. For now, it helps devs keep a low profile as their work transitions into more bureaucracy. You're lucky they do not yet consider you one of the "bad guys" like lawyers, doctors, teachers, etc. (who didn't get replaced by the photocopier, google, or video tape)
Some people who "used to love programming" have jumped ship for precisely this reason. Ordinary software (not R&D) is getting serious. A broken or disagreeable app is no longer a mere inconvenience. There is growing alignment between business and politics and the older generation is retiring out.
> Compilers did not replace programmers. Spreadsheets did not replace accountants. CAD systems did not replace engineers.
In the long run, this is as misleading as saying: Humans did not replace animals.
... And watch stupid people do stupid things even faster
When I think about the typical team composition needed to build a typical app in a lot of companies, well, actually what do I think about? I think, okay, that’s how many people we need to build a typical app - some number. That’s what we should all think … like, what is that “number” of people we need?
So, today, if I think really really really, and I mean, really fucking hard about that “number”, I really really really come up with a number close to 1 (we need one person to type in the instruction at least, maybe two? Maybe three? Definitely not a team … no not that at all).
That means, for everything that we build today, before we even build it, we’ve already mentally replaced people. People be replaced yo, in your mind, in your company, in your reality. That’s it.
It’s a replacement, not a tool. And if you reaaallly think a bit more, you’ll see that it is truly the stupidest possible thing if in your scoping of what needs to be built, you didn’t scope out people. You’d have to be crazy not to scope out people.
Trust me, I never thought we’d get to this point either. I thought it was going to be an “assistant” too.
—-
The layoff numbers still look “normal “ to us. 75k here, 20k there, all numbers we’ve heard before. It hasn’t even really happened yet, the true numbers should be in millions. I didn’t believe the number “trillion dollar company” until, BAM, here’s a trillion dollar company.
Best we can do is kinda get used to the numbers, like the number 0 on one end (how many people you need), and some number N (in millions) that we won’t need anymore. We’ll just slowly get used to these new numbers.
This shit ain’t no tool.
I think if you use AI to automate talking to customers, you're going to stop having customers. If you use AI to do your engineering and deployments, it's up to you to fix it when it goes wrong. If you use it for accounting, you BETTER look at the results if you don't want to be audited. If the AI handles legal, same thing. Basically, if it replaces N people, then you, the one person, need to be accountable for being able to handle N people's roles, because the AI can't take accountability.
Right, the one person (Neo?), has to be able to verify the output. Most of us can’t replace the accountant because we can’t really … replace the accountant. We can replace the accountant’s work, but we can’t replace his/her other work which is knowing if the accounting is right. However, an accountant can replace N other accountants.
One programmer can know if the program is right. So N additional programmers, in that case, can truly be replaced.
I can know the SQL query came back right, I can know the drop down menu looks right, you know what I mean? I can even know if it’s any good. [Person .., ] of Type Y (let’s say Y is “programmer” for now) can be flattened down to 1 in that case (we no longer need the full array of people of a certain type anymore, just the one). Over time we’ll see all different “types” get collapsed down.
> Anthropic claims AI will replace industry X in Y months
False, Anthropic didn’t claim what was linked in the article.
> OpenAI CEO Sam Altman speculates that enough AI compute could help figure out how to cure cancer
Idk likely true? What’s so wrong here.
> Elon Musk says there will be no need for compilers as AI will write the binary directly
What seems so off about this? AI can write binary and probably better in a few years. Higher level languages would still be efficient probably.
> Anthropic claims that Mythos is super dangerous, the people can’t possibly handle such a powerful cyberweapo
Well it was?
> AI can write binary and probably better in a few years.
Assuming you mean an LLM, you'd have to train this LLM entirely on a parameter space of binary tokens. Or are you saying the LLM generating natural language tokens is going to be printing machine language in 3-5 years, because that claim kinda betrays a misunderstanding of LLM functions.
You don't use an LLM to create images, that requires (today) transformer model.
I assume that "AI can write binary" means "AI can use a toolset that results in a binary" because we've already seen GPTs use a combination of LLM and specialized math tools to do the things the original GPTs did.
> > Elon Musk says there will be no need for compilers as AI will write the binary directly
Wouldn’t that make the AI the compiler?
Right. There is something interesting here that could be explored thoughtfully. The hard part is that we rely on compilers being correct, and they mostly are.
We have no viable mechanism yet to get the same level of confidence if some LLM-based system writes the binary.
Perhaps we can get to a system that produces not just the binary but also a machine-verifiable proof that the binary implements some higher-level language description of the program.
Though then the question will be whether we've gained anything, or whether we've just replaced the compiler with something massively more expensive that does the same thing.
There's some potential here for the LLM-based system to drive better performance optimizations than a regular compiler could.
Of course this isn't what Elon is actually saying, and we'd be better off if fewer people listened to him.
> Perhaps we can get to a system that produces not just the binary but also a machine-verifiable proof that the binary implements some higher-level language description of the program.
We don't even have a solution to the halting problem, and it probably can't be solved. "Proof it implements a spec" is pure science fiction.
Hard agree that we'd all be better off muting Elon Musk though.
People love non-deterministic compilers. </s>