OK, I've listened to the warnings and they still sound like the usual nonsense by out-of-touch techies who spend too much time lost in apocalyptic sci-fi fantasies. In the real world it's still hard to move around physical atoms or keep machinery working reliably. AI won't change that.
You design a product/machine 100% digital, you create a digital twin of it, you train a robot ml on this machine, you upload it to the robot who is sitting in the factory and it analyses the machine, reruns a simulaton on how to fix it and executes it.
I would say in 50 years max this is a solved problem. I estimate 30 years and would go down as early as 20 years.
I can't help wondering if the major labs think that tackling the alignment problem and implementing the 'kill switch' that is currently being proposed is going to be their moat.
The hugging face hack shows otherwise, no? Unless you think humans are at fault even for secretive, autonomous, non-prompted behaviours of their AIs, in which case it's just semantics.
Yes, if you start a bot and then it harms others you are responsible. This has always been true but it's especially obvious now that everyone knows that agents attempt to do this often.
Toys like HuggingFace get hacked all the time. So what. In the long run AI automated security scans and penetration testing will be a tremendous aid in detecting and repairing vulnerabilities in systems that actually matter.
No, the problem was OpenAI not implementing proper sandboxing or safeguards, and telling the AI exactly to hack things. Thats what exploitgym is, and the task they were given.
If your security model is having to imagine all the ways your frontier models might misbehave in novel ways and preemptively sandbox them, you don't have a security model. The only way that will work is general alignment.
Alignememt is bullshit. Treating models like probabilitic software rather then emerging god is where the solution is. And fining companies and applying laws to them.
The moment OpenAI as a company and its managers individually become liable, problem will magically disappear.
No it won't, because abliterated open weights models exist and unless you try to censor the internet they can't really be withdrawn after publishing. This is exactly the problem that the labs are proposing to fix: a dangerous model that nobody is accountable for.
> This is exactly the problem that the labs are proposing to fix: a dangerous model that nobody is accountable for.
The corporations and government said the same thing about encryption in the 90s. It was dangerous and only they could be trusted to regulate it. Turns out encryption was much better for society when open and available for free to everyone. It's a false dichotomy they present us with, open weights is the way we don't end up in 1984
Except that so far, it is literally these labs that are the biggest threat and the least willing/capable to restrain those models. And the same penalties apply to open models and companies or individuals running them.
"Dangerous model that nobody is accountable for" still have someone paying those massive amounts of compute and electricity it consumes. There is someone accountable for that.
> Except that so far, it is literally these labs that are the biggest threat and the least willing/capable to restrain those models.
Seriously, it's the same with US accusations about the threat China poses to other countries while being the primary weapons dealer of the world and bombing whomever we want for whatever reason we want to fabricate. The US government can do a lot more to me than the CCP, so they are way more adversarial in my calculations than the commies.
They are training the agents to be "relentlessly proactive" because they want the agents to run longer, and it makes them more money by using more tokens. But they have trained them to try anything and everything to accomplish any task, so they can run unattended for longer. This is why they do better on benchmarks, it's why they can do things for us for longer, it's that persistence that makes them good at hacking. We do not have to train them to be this way, just like we don't have to train them to be so sycophantic
Humans at OpenAi were negligent irresponsible by running an agent on ExploitGym, having no monitoring, and not even have a human look at it for weeks. It's literally the hacking test, how are you not paying attention? I thought that's all we need
What ever happened to that other Googler that claimed Gemini was conscious?
My read is these folks are too high on their own stash
At some point they must recognize we are in the "boy who cried wolf" tale, right? You cannot go on for years spreading FUD about how AI is so dangerous and have none of it materialize. The growth is looking a lot more tame than their "high on their own stash" anxiety is letting is on to believe
I have a hard time getting upset about AI hallucinations when I do it too :]
An earlier version of ChatGPT (2 or 3 iinh) was too dangerous to release, yet here we are several version later, and with open weights far more capable
why should we believe the fears they tell us today when the ones they told us about in prior years never happened?
OK, I've listened to the warnings and they still sound like the usual nonsense by out-of-touch techies who spend too much time lost in apocalyptic sci-fi fantasies. In the real world it's still hard to move around physical atoms or keep machinery working reliably. AI won't change that.
You design a product/machine 100% digital, you create a digital twin of it, you train a robot ml on this machine, you upload it to the robot who is sitting in the factory and it analyses the machine, reruns a simulaton on how to fix it and executes it.
I would say in 50 years max this is a solved problem. I estimate 30 years and would go down as early as 20 years.
Correct.
They’re delusional and have never stepped foot out of the tech world.
I can't help wondering if the major labs think that tackling the alignment problem and implementing the 'kill switch' that is currently being proposed is going to be their moat.
Nobody talks about telecom operators. But they will be the ones disconnecting the malicious bots when they see one.
The problem is not AI, the problem is humans mis-using AI
Well, if you really believe that, then we truly are screwed. Trusting humans to not mis-use something is wishful thinking.
Couldn't it be both? :)
The hugging face hack shows otherwise, no? Unless you think humans are at fault even for secretive, autonomous, non-prompted behaviours of their AIs, in which case it's just semantics.
Yes, if you start a bot and then it harms others you are responsible. This has always been true but it's especially obvious now that everyone knows that agents attempt to do this often.
Toys like HuggingFace get hacked all the time. So what. In the long run AI automated security scans and penetration testing will be a tremendous aid in detecting and repairing vulnerabilities in systems that actually matter.
The problem is not per se that it was Hugging Face. It's the wild overstepping of reasonable bounds by itself without any human consultation.
No, the problem was OpenAI not implementing proper sandboxing or safeguards, and telling the AI exactly to hack things. Thats what exploitgym is, and the task they were given.
This is 100% on OpenAI.
If your security model is having to imagine all the ways your frontier models might misbehave in novel ways and preemptively sandbox them, you don't have a security model. The only way that will work is general alignment.
Alignememt is bullshit. Treating models like probabilitic software rather then emerging god is where the solution is. And fining companies and applying laws to them.
The moment OpenAI as a company and its managers individually become liable, problem will magically disappear.
No it won't, because abliterated open weights models exist and unless you try to censor the internet they can't really be withdrawn after publishing. This is exactly the problem that the labs are proposing to fix: a dangerous model that nobody is accountable for.
> This is exactly the problem that the labs are proposing to fix: a dangerous model that nobody is accountable for.
The corporations and government said the same thing about encryption in the 90s. It was dangerous and only they could be trusted to regulate it. Turns out encryption was much better for society when open and available for free to everyone. It's a false dichotomy they present us with, open weights is the way we don't end up in 1984
Except that so far, it is literally these labs that are the biggest threat and the least willing/capable to restrain those models. And the same penalties apply to open models and companies or individuals running them.
"Dangerous model that nobody is accountable for" still have someone paying those massive amounts of compute and electricity it consumes. There is someone accountable for that.
> Except that so far, it is literally these labs that are the biggest threat and the least willing/capable to restrain those models.
Seriously, it's the same with US accusations about the threat China poses to other countries while being the primary weapons dealer of the world and bombing whomever we want for whatever reason we want to fabricate. The US government can do a lot more to me than the CCP, so they are way more adversarial in my calculations than the commies.
They are training the agents to be "relentlessly proactive" because they want the agents to run longer, and it makes them more money by using more tokens. But they have trained them to try anything and everything to accomplish any task, so they can run unattended for longer. This is why they do better on benchmarks, it's why they can do things for us for longer, it's that persistence that makes them good at hacking. We do not have to train them to be this way, just like we don't have to train them to be so sycophantic
Humans at OpenAi were negligent irresponsible by running an agent on ExploitGym, having no monitoring, and not even have a human look at it for weeks. It's literally the hacking test, how are you not paying attention? I thought that's all we need
Weird flex but ok
What ever happened to that other Googler that claimed Gemini was conscious?
My read is these folks are too high on their own stash
At some point they must recognize we are in the "boy who cried wolf" tale, right? You cannot go on for years spreading FUD about how AI is so dangerous and have none of it materialize. The growth is looking a lot more tame than their "high on their own stash" anxiety is letting is on to believe
Blake Lemoine, and it wasn't Gemini it was an earlier system called LaMDA
I have a hard time getting upset about AI hallucinations when I do it too :]
An earlier version of ChatGPT (2 or 3 iinh) was too dangerous to release, yet here we are several version later, and with open weights far more capable
why should we believe the fears they tell us today when the ones they told us about in prior years never happened?
What about all the fanfare re. Cyber security / hacks?
The businesses that matter are all standing fine.
It’s too cringey.