Off-topic, but this article brought my first time to see a "scan the QR to verify you're human" from reCaptcha. I hope this doesn't become a thing, because I never, ever do that.
The archive link given by another poster works for me.
What matters is the pessimistic conclusion of the article:
> "The uplift in prices is anticipated to persist for as long as the AI supercycle lasts. “I think we’re still in the sort of early stages [of price increases],” said Dhillon, adding that “there’s a lot more to come in the pipeline”."
Even if there is a cartel it's secondary now because Nvidia is basically vacuuming up half of Hynix's output for Blackwell and leaving the retail market to fight over the scraps
The economics of owning your own powerful hardware make sense less and less with each passing year. Compute that's rented from the cloud gets far more utilization per dollar spent, and that's just on the hardware costs, let alone the cost of electricity to run it. What we're seeing now is the market adjusting to this new reality, as foundries reallocate capacity from consumer-oriented hardware to gear aimed at superscalers.
I don't know.... With all those approaches to run AI in your own premise, people need their own hardware. It makes them independent. Of course hyperscalers and AI labs don't like that, but I argue that consumers prefer independence over some costs of operations. If this would be the case, people wouldn't own cars.
server -> personal -> server -> personal... I've seen this cycle a number of times in the last 40 years. Almost none of the people making any of the software we use are purely using the cloud, they are using powerful personal computers with cloud for GPU. The ubiquity of personal computing is required if we want this industry to keep producing enthusiastic hackers.
If the AI companies ever get the idea that it would make sense for the AI model to modify its KV cache, it would end up adding a write step to the KV cache per token (moderately expensive) and increase the cost of inference by the width of the update (arbitrarily expensive).
This in itself could double or triple the demand for both memory bandwidth and compute. I honestly don't see a future where AI demand disappears...
https://archive.is/RLhsn
Off-topic, but this article brought my first time to see a "scan the QR to verify you're human" from reCaptcha. I hope this doesn't become a thing, because I never, ever do that.
You can't even read a single line of the article at this link, its just a full page ad obscured by various popups.
The archive link given by another poster works for me.
What matters is the pessimistic conclusion of the article:
> "The uplift in prices is anticipated to persist for as long as the AI supercycle lasts. “I think we’re still in the sort of early stages [of price increases],” said Dhillon, adding that “there’s a lot more to come in the pipeline”."
What about the cartel? Have they punished Samsung and Hynix yet?
Even if there is a cartel it's secondary now because Nvidia is basically vacuuming up half of Hynix's output for Blackwell and leaving the retail market to fight over the scraps
I remember the HW shortage during 2020-2021, fun time.
This is going to last longer.
The economics of owning your own powerful hardware make sense less and less with each passing year. Compute that's rented from the cloud gets far more utilization per dollar spent, and that's just on the hardware costs, let alone the cost of electricity to run it. What we're seeing now is the market adjusting to this new reality, as foundries reallocate capacity from consumer-oriented hardware to gear aimed at superscalers.
I don't know.... With all those approaches to run AI in your own premise, people need their own hardware. It makes them independent. Of course hyperscalers and AI labs don't like that, but I argue that consumers prefer independence over some costs of operations. If this would be the case, people wouldn't own cars.
Yes but we are waiting fro the release of the right hardware to enable local inference efficiently.
These times are confused because all the available solutions are suboptimal for NN inference in the consumer market.
server -> personal -> server -> personal... I've seen this cycle a number of times in the last 40 years. Almost none of the people making any of the software we use are purely using the cloud, they are using powerful personal computers with cloud for GPU. The ubiquity of personal computing is required if we want this industry to keep producing enthusiastic hackers.
If the AI companies ever get the idea that it would make sense for the AI model to modify its KV cache, it would end up adding a write step to the KV cache per token (moderately expensive) and increase the cost of inference by the width of the update (arbitrarily expensive).
This in itself could double or triple the demand for both memory bandwidth and compute. I honestly don't see a future where AI demand disappears...