No doubt this will be used to justify a government-funded capital lifeline to OAI and Anthropic who are still bleeding cash (irrespective of whether inference is profitable) and may have issues raising more money in the private markets if they are signaling a delayed IPO.
The proposed “donation” of a 5% stake to a sovereign wealth fund creates a direct incentive for government cash infusion.
I really can’t begin to describe how angry this possibility makes me. And I don’t think I’m alone. Keep pushing the envelope Sam / Dario and see what it gets you. Doubling down on a losing bet just digs your hole deeper.
What happens when the government sinks half a trillion dollars into this and we still don’t see an ROI / true agent autonomy? Then what? Ask for another trillion dollars and hope you can stumble on a research breakthrough equally as revolutionary as the transformer?
What's jarring is nobody is asking the hard question - how much was expended to make the breakthrough of Transformers?
It certaintly was nothing close to half a trillion.
Clearly more money is not the path to the solution. Furthermore China is doing pretty well with a fraction of the spend. America may have money but money needs to go toward productive projects - this requires ideas and vision. Which cannot be bought actually.
Well absent a research breakthrough the only option is “scaling” which according to Sam Altman was logarithmic growth in model ability with number of weights.
Of course any stats undergrad could tell you this was a fairytale. Increasing number of weights only works until you exhaust the signal in the data. I’m not sure how he was allowed to get away with such a blatant lie but here we are.
This lie is effective because on a small time scale it’s impossible to tell the difference between logarithmic growth and logistic growth. If you maintain a fixed training data size, increasing the size of the model will get you logistic growth in model capability meaning that past a certain size you get effectively no gain in performance because you’ve already squeezed out 99% of the signal.
“The intelligence of an AI model roughly equals the log of the resources used to train and run it.”
He is playing loose with the language here because the only way this statement holds is when resources = breadth and depth of training data - not compute / model size.
The administration is very knowledgeable about all things A1.
Not true at all, I hate misinformation like this. They like their steak well done with ketchup on it. No A1 sauce in sight.
Your parent comment may be an example of Poe’s Law.
No doubt this will be used to justify a government-funded capital lifeline to OAI and Anthropic who are still bleeding cash (irrespective of whether inference is profitable) and may have issues raising more money in the private markets if they are signaling a delayed IPO.
The proposed “donation” of a 5% stake to a sovereign wealth fund creates a direct incentive for government cash infusion.
I really can’t begin to describe how angry this possibility makes me. And I don’t think I’m alone. Keep pushing the envelope Sam / Dario and see what it gets you. Doubling down on a losing bet just digs your hole deeper.
What happens when the government sinks half a trillion dollars into this and we still don’t see an ROI / true agent autonomy? Then what? Ask for another trillion dollars and hope you can stumble on a research breakthrough equally as revolutionary as the transformer?
What's jarring is nobody is asking the hard question - how much was expended to make the breakthrough of Transformers?
It certaintly was nothing close to half a trillion.
Clearly more money is not the path to the solution. Furthermore China is doing pretty well with a fraction of the spend. America may have money but money needs to go toward productive projects - this requires ideas and vision. Which cannot be bought actually.
Well absent a research breakthrough the only option is “scaling” which according to Sam Altman was logarithmic growth in model ability with number of weights.
Of course any stats undergrad could tell you this was a fairytale. Increasing number of weights only works until you exhaust the signal in the data. I’m not sure how he was allowed to get away with such a blatant lie but here we are.
This lie is effective because on a small time scale it’s impossible to tell the difference between logarithmic growth and logistic growth. If you maintain a fixed training data size, increasing the size of the model will get you logistic growth in model capability meaning that past a certain size you get effectively no gain in performance because you’ve already squeezed out 99% of the signal.
This disproves point number 1 in Sam’s thesis: https://blog.samaltman.com/three-observations
“The intelligence of an AI model roughly equals the log of the resources used to train and run it.”
He is playing loose with the language here because the only way this statement holds is when resources = breadth and depth of training data - not compute / model size.