Will the workloads exist? As in not going straight to something at least as dedicated as the Strix Halo CPU/GPU combo (with air quotes around the G)? Or the Apple max? Somehow I don't picture this as an attempt to make a full AI rig that just happens to be x86 in the housekeeping parts but just something that will make CPU inference a little less bad. In that case it would at best be a hedge against some low requirements use case becoming more important than expected, yet another unused spec sheet checkbox engineering marvel otherwise.
Not long ago, China has demonstrated a supercomputer faster than any existing US supercomputer.
The Chinese supercomputer uses custom CPUs designed in China, which implement the SME matrix extension of Armv9.3-A, while the fastest US supercomputers use GPUs.
The Chinese CPUs with SME have a lower energy efficiency than the best NVIDIA datacenter GPUs, and also a little lower than the best AMD datacenter GPUs, but not only they reached a greater absolute performance, but they also reach a much higher percentage of their theoretical maximum throughput, and this is likely to happen for all problems solved on them, because writing efficient programs is easier for CPUs than it is for GPUs.
As discussed in TFA, the Arm SME and the Intel/AMD ACE are similar.
Thus server CPUs that have hundreds of cores and ISAs with matrix extensions can be very competitive in performance with GPUs, even if it is very likely that GPUs will always hold the records for the best energy efficiency.
The Chinese CPUs have fast HBM memory, with an 8 TB/s memory bandwidth per socket, equal to that of the latest AMD Instinct datacenter GPU and much greater than that of most less expensive GPUs. Very few consumer GPUs exceed a 1 TB/s memory bandwidth.
So at least where electric energy is cheap and abundant, running AI inference or training on server CPUs may be a competitive solution, especially when that computing cluster is also used for other purposes, not only for AI.
Dedicated PCIe or memory-attached inference units will continue to exist, and will continue to do the heavy lifting, but an ISA extension provides latencies any external unit would have trouble matching. You can, with some work, extract some decent throughput with CPUs alone, with a large enough CPU you can use for non-AI jobs the rest of the time. There was a nice writeup not that long ago here on HN describing the flags and the reasoning behind them to use on that specific machine.
Yeah, I guess in these days of hardware not getting completely outdated even after half a decade, it might well make sense to spend chip area on specialist units that are shut down almost the entire time.
In the old days, when it was all about squeezing maximum performance from a given chip area budget you'd prefer using those transistors for some improvement of the general purpose parts, no matter how marginal. But now it's the thermal envelope, or power consumption of you prefer looking at it from that angle, and even slightly esoteric special purpose units can start making sense.
Dark silicon is also a thermal management strategy. Areal usage can go down, spreading heat generation over a larger space and making cooling the chip easier.
This all went over my head, but does anyone know either how much faster this will make things (4x faster than AVX512 at 2048-bit??), and if unified memory plus a basic GPU will render this dead in the water?
It's impossible to say what sort of improvement (if anything) this will give.
My assumption is the determining factor in performance is going to ultimately be memory bandwidth.
> and if unified memory plus a basic GPU will render this dead in the water?
Yeah, generally I think that is the case. The main benefit this thing would have is power consumption will likely be lower than what it is to send this operation to the GPU.
FWIW, I wouldn't be surprised if you only have a couple threads using this at a time max since it looks an awful lot Apple's AMX/SME stuff. Those Apple execution units only have single engines shared about per cluster.
XSAVE lets you not bother saving register state that user space hasn't changed at the granularity of each large feature.
Great article. What does the (organizational) process look like to convert one of these specs to a processor product, does it go through a committee like the C++ standards?
> AMD, in partnership with Intel and the x86 EAG (Ecosystem Advisory
Group) [EAG24], is readying ACE as the standard matrix acceleration architecture for x86, further enhancing the already
vibrant x86 ecosystem.
Now I wonder how many such "extension packages" Intel (and AMD?) have introduced since MMX (which I believe was the first, and interestingly enough didn't have an official meaning - https://en.wikipedia.org/wiki/MMX_(instruction_set)) in 1997?
Sounds interesting, I wonder if any of it will be available on consumer hardware. Making CUDA work across the stack seems to be the key thing that helped NVIDIA become the main purveyor of GPUs for AI. Apple seems to be doing well by the same approach.
Last time I looked, anything remotely useful here was only on the Xeon chips, which means there'll be very few open source and even fewer non-corporate folks doing anything interesting with it.
AMD seems to be better at that, maybe there's hope.
Well... They are registers, so yes. The question is whether it'd make sense to do it and what problem would that solve.
In a past life, I taught young programmers you can, of course, make a phone number a numeric field in your database (and save a couple bytes by doing that!) but it'd only make sense if you wante to figure out the average phone number for all your clients so you could call them all at once.
There are extreme cases where it would actually make sense, but that I usually left to dig into in a later discussion.
I don't know enough about phone numbers to say whether you could encode them as an int anyway. In the UK all standard numbers start 0 and international codes start 00? Of course you could introduce a 'locale' and then 20 years later try to solve that mess with multiple ways of representing international numbers. Or maybe not teach young programmers that you can make the phone number a numeric field?
So it seems to me the solution is to save them as BCD...
In addition to digits, conventions for phone numbers include the plus sign (international prefix), comma (dial pause), dashes or parentheses (zone/area codes), and spaces (logical separation). I don't see why you'd want to burden yourself with having to regenerate a locally-correct phone number representation each time it needs to be displayed to the user.
I think you and and your parent are in agreement, you CAN do something, but it's not useful, and then you have to either by convention or additional fields deal with the leading zeros and other non-numeric aspects of phone numbers.
I would also teach young programmers (about data) with the shorthand used above, things like "Do you want to add phone numbers? Multiply them? Utilize fractional components? Why are we storing these as numbers?"
I'm not sure we are. My parent suggested you could, in the sense of why would you. I'm questioning if it's possible at all. And if so, is that just based on a local solution. Ie all numbers have an implicit 0 at the start.
To me storing as a number to save space is a fine goal itself. The issue is the underlying complexity which a bare int can't capture. Suggesting using an int indicates the proposer hasn't thought about the issue.
This isn't to say the parent was wrong person se. Asking why you are doing something is a good question. But also understanding the issue/complexity is important too.
In the example I would say that saving space is a sufficient reason, so the point is moot. The reason why you wouldn't want to do it is because a phone number is not 'just a number' (I'm sure there's a mathematical term for a number where leading zeros are important etc).
So the UK starts phone numbers with a 0 generally. International numbers 00 (or plus)
You can't represent a leading zero and because there's 2 for an international code you either rely on the 2 not conflicting. So in the UK's case, the international code is 0044 you would have to rely on no other countries using 044 for domestic numbers.
You could get around that by using the sign bit so +44 is literally a positive integer and a non international prefix is negative?
Then there's the non standard phone numbers, eg emergency services. How do they play with all this. Does 0999 or 0911 work? Does that work internationally?
The biggest issue that I can't see around is that Australia's emergency number is 000. How do you encode that as an integer? How do you differentiate from the old US operator number of 0?
The problem is a leading 0 isn't nothing in a phone number. But it is in an integer on a computer.
So then we're left with the definition of possible. Maybe you can encode all the numbers in your particular region. That's possible. But you can't encode any legal phone number globally.
The use of "What could possibly go wrong" as a provocation, to make students explore what actually can go wrong when we provide a terrible idea (or a good one that has a couple terrible aspects hidden inside them)
Will the workloads exist? As in not going straight to something at least as dedicated as the Strix Halo CPU/GPU combo (with air quotes around the G)? Or the Apple max? Somehow I don't picture this as an attempt to make a full AI rig that just happens to be x86 in the housekeeping parts but just something that will make CPU inference a little less bad. In that case it would at best be a hedge against some low requirements use case becoming more important than expected, yet another unused spec sheet checkbox engineering marvel otherwise.
Not long ago, China has demonstrated a supercomputer faster than any existing US supercomputer.
The Chinese supercomputer uses custom CPUs designed in China, which implement the SME matrix extension of Armv9.3-A, while the fastest US supercomputers use GPUs.
The Chinese CPUs with SME have a lower energy efficiency than the best NVIDIA datacenter GPUs, and also a little lower than the best AMD datacenter GPUs, but not only they reached a greater absolute performance, but they also reach a much higher percentage of their theoretical maximum throughput, and this is likely to happen for all problems solved on them, because writing efficient programs is easier for CPUs than it is for GPUs.
As discussed in TFA, the Arm SME and the Intel/AMD ACE are similar.
Thus server CPUs that have hundreds of cores and ISAs with matrix extensions can be very competitive in performance with GPUs, even if it is very likely that GPUs will always hold the records for the best energy efficiency.
The Chinese CPUs have fast HBM memory, with an 8 TB/s memory bandwidth per socket, equal to that of the latest AMD Instinct datacenter GPU and much greater than that of most less expensive GPUs. Very few consumer GPUs exceed a 1 TB/s memory bandwidth.
So at least where electric energy is cheap and abundant, running AI inference or training on server CPUs may be a competitive solution, especially when that computing cluster is also used for other purposes, not only for AI.
Dedicated PCIe or memory-attached inference units will continue to exist, and will continue to do the heavy lifting, but an ISA extension provides latencies any external unit would have trouble matching. You can, with some work, extract some decent throughput with CPUs alone, with a large enough CPU you can use for non-AI jobs the rest of the time. There was a nice writeup not that long ago here on HN describing the flags and the reasoning behind them to use on that specific machine.
Funny thing, I can't find the article.
Yeah, I guess in these days of hardware not getting completely outdated even after half a decade, it might well make sense to spend chip area on specialist units that are shut down almost the entire time.
In the old days, when it was all about squeezing maximum performance from a given chip area budget you'd prefer using those transistors for some improvement of the general purpose parts, no matter how marginal. But now it's the thermal envelope, or power consumption of you prefer looking at it from that angle, and even slightly esoteric special purpose units can start making sense.
Dark silicon is also a thermal management strategy. Areal usage can go down, spreading heat generation over a larger space and making cooling the chip easier.
I assume Zen 6 won't support these, so we are looking Zen 7 at the earliest, which is 2028 earliest.
In the meantime x86 don't have much in the roadmap that compete well with ARM vendor's offering. And that was before Nvidia decided to join the fight.
This all went over my head, but does anyone know either how much faster this will make things (4x faster than AVX512 at 2048-bit??), and if unified memory plus a basic GPU will render this dead in the water?
It's impossible to say what sort of improvement (if anything) this will give.
My assumption is the determining factor in performance is going to ultimately be memory bandwidth.
> and if unified memory plus a basic GPU will render this dead in the water?
Yeah, generally I think that is the case. The main benefit this thing would have is power consumption will likely be lower than what it is to send this operation to the GPU.
With 8kb of registers for just this one feature, what does a modern process-control-block look like?
On x86, basically whatever XSAVE writes out.
FWIW, I wouldn't be surprised if you only have a couple threads using this at a time max since it looks an awful lot Apple's AMX/SME stuff. Those Apple execution units only have single engines shared about per cluster.
XSAVE lets you not bother saving register state that user space hasn't changed at the granularity of each large feature.
Great article. What does the (organizational) process look like to convert one of these specs to a processor product, does it go through a committee like the C++ standards?
> AMD, in partnership with Intel and the x86 EAG (Ecosystem Advisory Group) [EAG24], is readying ACE as the standard matrix acceleration architecture for x86, further enhancing the already vibrant x86 ecosystem.
Source: https://x86ecosystem.org/wp-content/uploads/2026/03/ACE-Whit...
"Vibrant" is certainly one way to describe it...
AI Compute Extensions, ACE.
Also, AMX = Advanced Matrix Extensions.
Now I wonder how many such "extension packages" Intel (and AMD?) have introduced since MMX (which I believe was the first, and interestingly enough didn't have an official meaning - https://en.wikipedia.org/wiki/MMX_(instruction_set)) in 1997?
https://www.sandpile.org/x86/cpuid.htm#leaf_0000_0001h_more
Then click "...details for the "generations" of Intel's vector instructions."
Beware, it's dense acronym soup. :)
Sounds interesting, I wonder if any of it will be available on consumer hardware. Making CUDA work across the stack seems to be the key thing that helped NVIDIA become the main purveyor of GPUs for AI. Apple seems to be doing well by the same approach.
Last time I looked, anything remotely useful here was only on the Xeon chips, which means there'll be very few open source and even fewer non-corporate folks doing anything interesting with it.
AMD seems to be better at that, maybe there's hope.
Could the registers potentially be used to store encryption keys?
Well... They are registers, so yes. The question is whether it'd make sense to do it and what problem would that solve.
In a past life, I taught young programmers you can, of course, make a phone number a numeric field in your database (and save a couple bytes by doing that!) but it'd only make sense if you wante to figure out the average phone number for all your clients so you could call them all at once.
There are extreme cases where it would actually make sense, but that I usually left to dig into in a later discussion.
How do you represent a leading 0?
I don't know enough about phone numbers to say whether you could encode them as an int anyway. In the UK all standard numbers start 0 and international codes start 00? Of course you could introduce a 'locale' and then 20 years later try to solve that mess with multiple ways of representing international numbers. Or maybe not teach young programmers that you can make the phone number a numeric field?
So it seems to me the solution is to save them as BCD...
In addition to digits, conventions for phone numbers include the plus sign (international prefix), comma (dial pause), dashes or parentheses (zone/area codes), and spaces (logical separation). I don't see why you'd want to burden yourself with having to regenerate a locally-correct phone number representation each time it needs to be displayed to the user.
I think you and and your parent are in agreement, you CAN do something, but it's not useful, and then you have to either by convention or additional fields deal with the leading zeros and other non-numeric aspects of phone numbers.
I would also teach young programmers (about data) with the shorthand used above, things like "Do you want to add phone numbers? Multiply them? Utilize fractional components? Why are we storing these as numbers?"
I'm not sure we are. My parent suggested you could, in the sense of why would you. I'm questioning if it's possible at all. And if so, is that just based on a local solution. Ie all numbers have an implicit 0 at the start.
To me storing as a number to save space is a fine goal itself. The issue is the underlying complexity which a bare int can't capture. Suggesting using an int indicates the proposer hasn't thought about the issue.
This isn't to say the parent was wrong person se. Asking why you are doing something is a good question. But also understanding the issue/complexity is important too.
In the example I would say that saving space is a sufficient reason, so the point is moot. The reason why you wouldn't want to do it is because a phone number is not 'just a number' (I'm sure there's a mathematical term for a number where leading zeros are important etc).
> I'm questioning if it's possible
It’s certainly possible, but you’ll regret it. Deeply.
Recognising terrible ideas is an essential skill for software engineers.
So the UK starts phone numbers with a 0 generally. International numbers 00 (or plus)
You can't represent a leading zero and because there's 2 for an international code you either rely on the 2 not conflicting. So in the UK's case, the international code is 0044 you would have to rely on no other countries using 044 for domestic numbers.
You could get around that by using the sign bit so +44 is literally a positive integer and a non international prefix is negative?
Then there's the non standard phone numbers, eg emergency services. How do they play with all this. Does 0999 or 0911 work? Does that work internationally?
The biggest issue that I can't see around is that Australia's emergency number is 000. How do you encode that as an integer? How do you differentiate from the old US operator number of 0?
The problem is a leading 0 isn't nothing in a phone number. But it is in an integer on a computer.
So then we're left with the definition of possible. Maybe you can encode all the numbers in your particular region. That's possible. But you can't encode any legal phone number globally.
“But why would you need to keep phones from other countries? All your clients will always be under your own area code.”
I used to make comments like that to highlight the absurdity of all this. I have a Leslie Nielsen level of deadpan delivery.
I read it as a Socraticish anti-question - answering it will answer why its a bad idea.
The use of "What could possibly go wrong" as a provocation, to make students explore what actually can go wrong when we provide a terrible idea (or a good one that has a couple terrible aspects hidden inside them)
Encode every 0 as 10 - then decode every 10 back to 0. Or more efficiently, the first 0 as 10, then decode the the first 10 as 0.
Add 10 to every digit, then multiply it by 10. Concatenate the numbers into the integer you will store.
I love to play absurd games.
Questionable acronym choice; immediately made me think of arbitrary code execution (which, you know, I would hope any processor would be capable of)