Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It seems like that would also have some power delivery advantages.
If you look at a any of the nvidia DGX boards it's already pretty close.
PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...
There are companies that specialize in memory controller ip that every one else uses, including large semi companies like Intel.
The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.
These days you can buy any standard as a soft IP from Synopsys or Cadence. They take their previous serdes and modify it to meet the new standard. They have thousands of employees across the globe just doing that.
Good to see I’m not the only person that’s been thinking about this. Wedging gargantuan GPUs onto boards and into cases, sometimes needing support struts even, and pumping hundreds of watts through a power cable makes little sense to me. The CPU, RAM, these should be modules or cards on the GPU. Imagine that! CPU cards might be back..
This is a legitimate problem in datacenters. They're getting to the point where a single 40(ish)OU/RU rack can pull a megawatt in some hyperdense cases. The talk of GPU/AI datacenters consuming inordinate amounts of energy isn't just because the DC's are yuge, (although some are), but because the power draw per rack unit space is going through the roof as well.
On the consumer side of things where the CPU's are branded Ryzen or Core instead of Epyc or Xeon, a significant chunk of that power consumption is from the boosting behavior they implement to pseudo-artificially[0] inflate their performance numbers. You can save huge (easily 10%, often closer to 30%, but really depends on exact build/generation) on energy by doing a very mild undervolt and limiting boosting behavior on these cpus and keeping the same base clocks. Intel 11th through 14th gen CPU's are especially guilty of this, as are most Threadripper CPU's. you can often trade single digit or even negligible performance losses (depends on what you're using it for and how much you undervolt/underclock/restrict boosting) for double digit reductions in power usage. This phenomenon is also true for GPU's when compared across the enterprise/consumer divide, but not quite to the significant extent in most cases.
Point being, yeah, it's a problem in data centers, but honestly there's a lot of headroom still even if you only have your common American 15A@120VAC outlets available before you need to call your electrician and upgrade your panel and/or install 240VAC outlets or what have you.
0: I say pseudo-artificial because the performance advantages are real, but unless you're doing some intensive/extreme cooling, they aren't sustainable or indicative of nominal performance, just a brief bit of extra headroom before your cooling solution heat-soaks and the CPU/GPU's throttle themselves back down. But it lets them put the "Bigger number means better" on the box for marketing.
It's not just about better numbers. Getting high clocks for a short period helps in a lot of use cases - say random things like a search. If I'm looking for some specific phrase in my codebase in vscode, everything spins up for the second or two it takes to process that.
Boosting from 4 to 5,5.5 ghz for that brief period shaves a fraction of a second - repeat that for any similar operation and it adds up.
Yes, I figured that much would be obvious to this crowd. Thus the "pseudo" part.
The point isn't that there isn't a benefit, it's that you start to pay exponentially more energy per 0.1GHz at a certain point. Furthermore, AMD and Intel were exceptionally aggressive about it in the generations I outlined (AMD would be 7000 series ryzens specifically), leading to instability issues on both platforms due to their spec itself being too aggressive, or AIB partners improperly implementing that spec as the headroom that typically exists from factory stock to push clocks/voltages further was no longer there in some silicon (some of it comes down to silicon lottery and manufacturing defects/mistakes (Intel's oxidation issues for example) but we're really getting into the weeds on this already)
And to clarify: I'm talking specifically of Intel turboboost and AMD's PBO boosting technologies where they boost where they boost well over base clocks, separate from the general dynamic clocking behavior where clocks will drop well below base when not in (heavy) use.
They're small and efficient, that means they can pack large numbers of those into small spaces, resulting in a similar large power draw per volume occupied by equipment in the DC. This is especially true with Apple's "Ultrafusion" tech which they're developing as quasi-analog to Nvidia Grace (Hopper) superchips.
Didn't saw they draw the same, I openly acknowledge their more efficient. Said power user per rack unit is trending up. This is true of Apple DC's as well, especially with their new larger/fused chip initiatives. It's an universal industry trends especially with AI compute, and Apple is not immune.
Let me rephrase to: No, they (collectively) don’t draw the same levels of power. I know what amperage is drawn by each rack. It’s nowhere near as much as was drawn by the older intel-based racks.
Changing settings can lead to stability issues no matter which way you push it frankly. If you're don't know what you're doing/aren't comfortable with it, probably not worth it.
At least with U.S. wiring we have 15 amps at 120 volts. For continuous power draw I know you'd want an 80% margin of safety, so let's say you have 1440 Watts of AC power you can safely draw continuously. Power supplies built on MOSFETs seem to peak at around 90% efficiency, but you could consider something like the Corsair AX1600i using gallium nitride transistors, which supposedly can handle up to 1600 watts at 94% efficiency.
Apparently we still have room, as long as you don't run anything else on the same circuit. :)
You can always have an electrician install a larger breaker for a particular circuit. I did that with my "server" area in my study, which was overkill cuz I barely pull 100w on it. But it cost nearly zero extra since he was doing a bunch of other things around the house anyway.
In older houses, made from brick and concrete, that can be tricky to do. The only reason I can have my computer on a separate circuit is because we could repurpose the old three phase wiring for a sauna we ripped out. If that had not been the case, getting the wires to the fuse board would have been tricky at best.
New homes are probably worse than old homes through. The wires a just chucked in the space been the outer and inner walls, there's basically no chance of replacing them of pulling new ones. Old houses at least frequently have piping in which the wires run.
The voltage is always going to be the same because the voltage is determined by the transformers leading to your service panel. The breakers break when you hit a certain amperage for a certain amount of time, so by installing a bigger breaker, you allow more amperage.
If you actually had an electrician do it, I doubt they would've installed a breaker if they thought the wiring wasn't sufficient. Truth is that you can indeed get away with a 20A circuit on 14 AWG wire if the run is short enough, though 12 AWG is recommended. The reason for this is voltage drop; the thinner gauge wire has more resistance, which causes more heat and voltage drop across the wire over the length of it, which can cause a fire if it gets sufficiently hot. I'm not sure how much risk you would put yourself in if you were out-of-spec a bit, but I wouldn't chance it personally.
You can, 240V on normal 12/2 Romex is fine. The neutral needs to be "re-labeled" with tape at all junctions to signify that it's hot, and then this practice is (generally) even code compliant.
However! This strategy only works if the outlet was the only one on the circuit, and _that_ isn't particularly common.
Although this exists, as a layperson, I've rarely seen it. There is the NEMA 6-15R receptacle type, but I have literally none of those in my entire house, and I've really never seen them. Apparently they're sometimes used for air conditioners. Aside from the very common 5-15R, I see 5-20R (especially in businesses/hospitals), and 14-30R/14-50R for ranges and dryers. (I have one for my range, but here in the midwest electric dryers and ranges aren't as common, so you don't always come across these here. We have LNG ran to most properties.) So basically, I just really don't see a whole lot of NEMA 6 receptacles. The NEMA 14 receptacles, though, require both hots and the neutral, so in a typical U.S. service panel it requires a special breaker and to take up two slots, so definitely not as simple of a retrofit.
(Another outlet type I've seen: I saw a NEMA 7 277V receptacle before. I think you get this from one phase of a 480V three-phase system, which I understand is ran to many businesses.)
If you drive an electric car in a rural area you might want to carry around 6-30 and 6-50 adapters because most farms have welders plugged into those and that can give you a quick charge. And also TT-30 and 14-50 adapters to plug in at campgrounds.
NEMA 6 is limiting because there’s no neutral, so everything in the device has to run on 240V. Your oven and dryer want 120V to run lights and electronics, so they use a 14 (or 10 for older installs) which lets them get 120V between a hot and the neutral.
Oddly, 14-50 has become the most common receptacle for non-hardwired EV charging, which is rather wasteful since EV charging doesn’t need the neutral at all. 6-50 would make more sense there.
Reasons why it's nice to have a 14-50 plug in your garage rather than a 6-50:
1: when an uncle stops by for a visit with his RV he can plug in.
2: the other outlets in your garage are likely on a shared circuit. The 14-50 is dedicated, so with a 14-50 to 5-15 adapter you can more safely plug in a high wattage appliance, like a space heater.
1 is why we ended up with 14-50 as the standard, too. Before there was much charging infrastructure, RV parks were a good place to get a semi-fast charge, and that meant a charger with a 14-50 plug.
2 is something I never thought of, I’ll have to keep that in mind.
NEMA 6s are extremely common in barns and garages for welders. 6-50 is more common for bigger welders but I’ve also seen 6-20s on repurposed 12/2 Romex as the parent post was discussing used for cheap EV retrofits, compressors, and welders.
If this is north america we're talking about, then 14 gauge is the standard for 120V 15A household circuits. By code, 20A requires 12 gauge. You'll notice the difference right away, it's noticeably harder to bend. Normally a house or condo will only have 15A wires running to circuits in the room. It's definitely not a standard upgrade, the 12 gauge wire costs a lot more per foot, no builder will do it unless the owner forks over extra dough.
Unless you performed the upgrade yourself or know for a fact that the wiring was upgraded to 12 gauge, it's very risky to just upgrade the breaker. That's how house fires start. It's worth it to check. If you know which breaker it is, you can see the gauge coming out. It's usually written on the wire.
I was actually under the impression that it is allowed depending on the length of the conductor, but it seems you are right. The NEC Table 15(B)(16) shows the maximum allowed ampacity of 14 AWG cables is 20 amperes, BUT... there is a footnote that states the following:
> * Unless otherwise specifically permitted elsewhere in this Code, the overcurrent protection for conductor types marked with an asterisk shall not exceed 15 amperes for No. 14 copper, 20 amperes for No. 12 copper, and 30 amperes for No. 10 copper, after any correction factors for ambient temperature and number of conductors have been applied.
I could've sworn there were actually some cases where it was allowed, but apparently not, or if there is, I'm not finding it. Seems like for 14 AWG cable the breaker can only be up to 15 amperes.
There is a chance he did not run new wires if he was able to ascertain that the wire gauge was sufficient to carry 20 amps over the length of the cable. This is a totally valid upgrade though it does obviously require you to be pretty sure you know the length of the entire circuit. If it was Southwire Romex, you can usually tell just by looking at the color of the sheathing on the cable (usually visible in the wallboxes.)
As an old house owner, I can attest to that for sure. In fairness though, I suspect most of the atrocities occur in wall and work boxes, as long as your house is new enough to at least have NM sheathed wiring instead of ancient weird stuff like knob and tube. That's still bad but it's a solvable problem.
I've definitely seen my share of scary things. I have a lighting circuit that is incomprehensibly wired and seems to kill LED bulbs randomly during a power outage; I have zero clue what is going on with that one. Also, often times opening up wall boxes I will see backstabs that were not properly inserted or wire nuts that are just covering hand-twisted wires and not actually threaded at all (and not even the right size in some cases...) Needless to say, I should really get an electrician in here, but at least with a thermal camera you can look for signs of serious problems.
Hair dryers and microwaves only run for a few minutes, so even if you do have too much resistance this probably won't immediately reveal a problem. A space heater might, but most space heaters I've come across actually seem to draw not much over 1,000 watts.
And even then, even if you do run something 24/7 at max wattage, it's definitely not guaranteed to start a fire even if the wiring is bad. Like, as long as it's not egregiously bad, I'd expect that there's enough margin to cover up less severe issues in most cases. I'm guessing the most danger would come when it's particularly hot outside (especially since then you'll probably have a lot of heat exchangers running.)
That's still not much for wiring in most countries. A small IKEA consumer oven is only 230V16A=3860W. Those GPUs and CPUs only consume that much at max usage anyway. And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
All American households get mains power at 240v (I'm missing some nuance here about poles and phases, so the electrical people can correct my terminology).
It's often used for things like ACs, Clothes Dryers, Stoves, EV Chargers.
So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
To get technical -- US homes get two phases of 120v that are 180 degrees out of phase with the neutral. Using either phase and the neutral gives you 120v. Using the two out of phase 120v phases together gives you a difference of 240v.
Yeah that's right. The grid is three phases (as it is basically everywhere in the world), and the transformer at the pole splits one of those in half. Although, what are technically half-phases are usually just called "phases" when they're inside of a home.
> So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
It'd be all new wire run (120 is split at the panel, we aren't running 240v all over the house) and currently electricians are at a premium so it'd likely end up costing a thousand+ to run that if you're using an electrician, more if there's not clear access from an attic/basement/crawlspace.
Though I think it's unlikely we'll see an actual need for it at home, I imaging a 800w cpu is going to be for server class CPUs and rare-ish to see in home environments.
I kinda suspect there’s a premium once you mention “EV vehicle”, since you’re signalling that you’re affluent enough to afford an EV and have committed to spending the money required to get EV charging at home working, etc. (Kinda like getting a quote for anything wedding related.)
I’m getting some wiring run about the same distance (to my attic, fished up a wall, with moderately poor access) for non-EV purposes next week and the quote was a few hundred dollars.
the trick is to request 240v outlet for welder. it brings price down to 400 or so.
running to another room will be done usually (at least in usa) through attic or crawlspace. i got it done a few months ago to have dedicated 20A circuit (for my rack) in my work room. cost was around 300-400 as well
Labor chargers alone are going to be higher than that in Seattle. Just to have someone come out on a call is going to be 150-200. If it is an independent electrician who owns their own business, maybe 100-150/hr, if they are part of a larger company, I'd expect even more than that.
Honestly I wouldn't expect to pay less than $1000 for the job w/o any markups.
Handy man prices around here are $65 to $100/hr, and there is a huge wait list for the good ones.
I've gotten multiple quotes on running the 240v line, the labor breakdown was always over $400 alone. Just having someone show up to do a job is going to be almost $200 before any work is done.
When I got quotes from unlicensed people, those came in around $1000 even.
in bayarea subreddits there are multiple posts talking about EV charger vs welder outlet and how it drops price from 2000 to 500 or so (depends on complexity).
another thing, which is good long term is to a find a local electrician (plumber, etc) who doesn't charge service calls and have reasonable pricing.
no idea about handyman pricing. never used any. for electrical/water/roofing i prefer somebody who is licensed/insured/bonded/etc
I should look at the label (or check with a meter..), but when I run my SGI Octane with its additional XIO SCSI board in active use, the little "office" room gets very hot indeed.
If we're counting all the phases then european homes get 400V 3-phase, not 240V split-phase.
Not that typical residential connections matter to highend servers.
Well yes its possible but often $500-1000 to run a new 240v outlet, and that's to a garage for an ev charger. If you want an outlet in the house I dont know how much wall people want to tear up and extra time and cost.
In the Nordics we're on 10A for standard wall outlets so we're stuck on 2300W without rewiring (or verifying wiring) to 2.5mm2.
We rarely use 16A but it exists. All buildings are connected to three phases so we can get the real juice when needed (apartments are often single phase).
I'm confident personal computers won't reach 2300W anytime soon though
In Italy we also have 10A and 16A (single phase). In practice however almost all wires running in the walls are 2.5 mm^2, so that you can use them for either one 16A plug or two adjacent 10A plugs.
In the Nordics (I'm assuming you mean Nordic countries) 10A is _not_ standard. Used to be, some forty years ago. Since then 16A is standard. My house has a few 10A leftovers from when the house was built, and after the change to TN which happened a couple of decades ago, and with new "modern" breakers, a single microwave oven on a 10A circuit is enough to trip the breaker (when the microwave pulses). Had to get the breakers changed to slow ones, but even those can get tripped by a microwave oven if there's something else (say, a kettle) on the same circuit.
16A is fine, for most things. 10A used to be kind of ok, with the old IT net and old-style fuses. Nowadays anything under 16A is useless for actual appliances. For the rest it's either 25A and a different plug, or 400V.
There already are different outlets for these higher power draw beasts in data centers. The amount of energy used in a 4u "AI" box is what an entire rack used to draw. Data centers themselves are having to rework/rewire areas in order to support these higher power systems.
You can up the voltage to 240 and re-use the wiring (with some minor mods to the ends), for double the power. Insulation class should be sufficient. That makes good sense anyway. You may still have an issue if the powersupply can't handle 240/60 but for most of the ones that I've used that would have worked. Better check with the manufacturer to be sure though. It's a lot easier and faster than rewiring.
Kettles in the US are usually 1500W, as the smallest branch circuits in US homes support 15A at 120V and the general rule for continuous loads is to be 80% of the maximum.
Ah, 16A at 230v (3680W) is a normal circuit here. Most appliances work with that, the common exception is electric cooking (using two circuits or 380v two-phase) and EV charging.
But computers do, which was why I included that context. You don't really want to build consumer PC >1500W in the US or you'd need to start changing the plug to patterns that require larger branch circuits.
Microwave ovens have a different issue, which I found when I upgraded my breaker board to a modern one in my house. The startup pulse gives a type of load which trips a standard A-type 10A breaker (230V). Had to get those changed to a "slow" type, but even that will trip every blue moon, and if there's something else significant on the same circuit the microwave oven will trip even so, every two weeks or so (for the record, I have several different types of microwave ovens around the house, and this happens everywhere there's a 10A circuit).
The newer circuits in the house are all 16A, but the old ones (very old) are 10A. A real pain, with new TN nets and modern breakers.
Microwave ovens top out around 1100-1250W output from a ~1500W input from the wall. Apparently there's a fair bit of energy lost in the power supply and magnetron that doesn't make it into the box where the food is.
It is mostly an issue in countries with 120V mains (I know that in the US 240V outlets exist though).
In France for example it is required that standard plugs must be able to deliver at least 16A on each outlet, at the 230V used here, we get 3600W of power, that’s more than enough.
Yes and this is something I've been thinking about for awhile.
A computer is becoming a Home Appliance in the it will need 20A wiring and plugs soon, but should move to 220/240v soon anyway (and change the jumper on your standard power supply).
But all of the most-ridiculous hyperscale deployments, where bandwidth + latency most matter, have multiple GPUs per CPU, with the CPU responsible for splitting/packing/scheduling models and inference workloads across its own direct-attached GPUs, providing the network the abstraction of a single GPU with more (NUMA) VRAM than is possible for any single physical GPU to have.
How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P
No, for a gaming computer what we need is the motherboard and gpu to be side by side. That way the heat sinks for the CPU and GPU have similar amounts of space available.
For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.
And the memory should be a onboard module on the cpu card intel/amd should replicate what apple did with a unified same ringbus sort of memory modules. Lower latency,higher throughput.
Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.
Yep, I have a lot of experience with CXL devices and networked PCIe/NVMe (over Eth/IB) Fabrics and deploying "Headless"/"Micro-Head" compute units which are essentially just a pair of DPU's on a PCIe multiplexer (basically just a bunch of PCIe slots tied to a PCIe Switch or two).
That said my experience in this field is more with storage than GPU compute, but I have done some limited hacking about in the GPGPU space with that tech as well. Really fascinating stuff (and often hard to keep up with and making sure every part in the chain supports the features you want to leverage, not to mention going down the PCIe root topology rabbit hole and dealing with latency/trace-length/SnR issues with retimers vs muxers vs etc etc etc).
It's still a nascent field that's very expensive to play in, but I agree it's the future of at least part of the data infrastructure field.
Really looking forward to finally getting my hands on CXL3.x stuff (outside of a demo environment.)
I've wondered why there hasn't been a desktop with a CPU+RAM card that slots into a PCIe x32 slot (if such a thing could exist), or maybe dual x16 slots, and the motherboard could be a dumb backplane that only connected the other slots and distributed power, and probably be much smaller.
If I remember correctly the military / aerospace shy away from this spec because the connector with the pins is on the backplane, with the sockets on the cards.
So if you incorrectly insert a card and bend a pin you're in trouble.
VPX has the sockets on the backplane so avoids this issue, if you bend pins you just grab another card from spares.
This may have changed since I last looked at it.
Telecoms industry definitely seem to favour TCA though.
Yes, for fucks sake, this is the only way forward. It gives us the ultimate freedom to do whatever we want in the future. Just make everything a card on the bus and quit with all this hierarchy nonsense.
Wouldn't that mean an complete mobo replacement to upgrade the GPU? GPU upgrades seem much more rapid and substantial compared to CPU/RAM. Each upgrade would now mean taking out the CPU/RAM and other cards vs just replacing the GPU
> GPU upgrades seem much more rapid and substantial compared to CPU/RAM.
I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.
Figure out how much RAM, L1-3|4 cache, integer, vector, graphics, and AI horsepower is needed for a use-case ahead-of-time and cram them all into one huge socket with intensive power rails and cooling. The internal RAM bus doesn't have to be DDRn/X either. An integrated northbridge would deliver PCIe, etc.
I wonder how many additional layers are required in the PCB to achieve this + how this will dramatically affect the TDP; the GPU's aren't the only components with heat tolerance and capacitance.
One possible advantage of this approach that no one here has mentioned yet is that it would allow us to put RAM on the CPU die (allowing for us to take advantage of the greater memory bandwidth) while also allowing for upgradable RAM.
GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds.
It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory?
It's a whole new architecture but it seems to get at the actual problems we have in the space.
I love the PCIe standard is 3 generations ahead of what is actually released. Gen5 is the live version, but the team behind it is so well organized that they have a roadmap of 3 additional versions now. Love it.
"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today, PCIe 7.x was finalized last month, and this is an announcement work on PCIe 8.0 has started for release in 3 years. I.e. it has only been one month of being one generation behind the latest PCIe revision.
It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.
either I'm misinformed in which case is like to see proof, or you are, but GB200 is not Gen6, just Gen5 and this will only be rectified with GB300 which, while exists in the wild, is not in the millions.
so indeed the parent commenter would be correct that everything is Gen5 right now.
At least the GB200s OpenAI purchased have CX-8s, but it's very possible (perhaps even very likely) most of the millions of GB200s deployed PCIe 5.0 using CX-7s given the SuperMicro spec sheet.
CPUs themselves only have so many PCI-E lanes though right? Wouldnt it make sense (even for consumers) to have peripherals using less lanes (but more speed per lane) for a multi GPU system or something that uses a lot of drives?
The chipset also strikes some of the balance for consumers though. It has a narrow high speed connection to the CPU but enables many lower speed devices to share that bandwidth. That way you can have your spare NVMe drive, SATA controller, wired and wireless NICs, sound hardware, most of your USB ports, your capture card, and some other random things connected over a single x4 to x8 sized channel. This leaves the high cost lanes for just the devices that actually use them (GPU and primary, possibly secondary, storage drive). I've got one consumer type Motherboard with 14 NVMe drives connected, for example, just not at full native speed directly to the CPU.
You're just SoL if you want to connect a bunch of really high bandwidth devices simultaneously (100 Gbps+ NICs, multiple GPUs at full connection speed, a dozen NVMe drives at native speed, or similar) because then you'll be paying for a workstation/server class platform which did make the "more faster lanes" tradeoff (plus some market segment gouging, of course).
One issue is that, at least on cheaper mobos, these don't work as a "total bandwidth budget" situation. And, especially with newer generation PCIe, it can be a bit frustrating.
Many mobos will operate the available slots such that the total number of active lanes is split between them. But if you use older-generation cards, you'll only get a fraction of the available bandwidth because you're only using a fraction of their lanes, although the physical lanes are physically present.
What I'm thinking about is something like, say, a pair of Gen3 NVMe drives that are good enough for mass storage (running in RAID-1 for good measure) and some cheap used 10 Gb NIC, which will probably be 8x Gen2, all running on a Gen4+ capable mobo.
And, while for a general-purpose setup I can live with splitting available BW between the NIC and the GPU (I most likely don't care about my download going super fast while I game), the downloads will generally go to the storage, so they must be fast at the same time.
Those MBs are cheaper precisely because supporting this kind of bandwidth breakout adds cost (-> a fancier PCIe switch in a higher end chipset/southbridge). If you add the support to do this to them you end up with the more expensive motherboard. Some of the highest end motherboards actually have 2 chipsets/PCIe switches, more cost but a more bandwidth sharing for the same number of lanes coming from the CPU.
You can also buy external PCIe switches (just make sure you're not accidentally buying a PCIe bifurcation device). Most of the time it's cheaper to just buy the higher end motherboard though, e.g. I don't want to know what price "Request a quote" for this PCIe switch which can do x8 4.0 upstream and then quad 4x 3.0 downstream https://www.amfeltec.com/pci-express-gen-4-carrier-board-for... is. I do have a few 3.0 era cards which were more reasonably priced though https://www.aliexpress.us/item/3256801702762036.html?gateway... and they've worked well for me.
How high-end are we talking about? Do you know off-hand of any models supporting this?
I haven't seen such features on boards under 200 EUR, from Asus, Asrock and Gigabyte.
The thing is, if I have to splurge for some 400 EUR "gaming" model, I might as well move to a "workstation" CPU supporting more lanes out of the box, and the mobo will be priced roughly the same.
Does the second half not already talk to this? Adding on to it further, GN found a few % performance difference on a 5090 going from PCIe 3.0 to 5.0 https://gamersnexus.net/gpus/nvidia-rtx-5090-pcie-50-vs-40-v... meanwhile many DC use cases find PCIe 5.0 unusably slow, driving alternative (non-PCIe) interconnects.
It takes a long time to get form standard to silicon, so I bet there are design teams working on pcie7 right now, which won't see products for 2 or more years
When AMD introduces a new Desktop CPU series IIRC they claim the next generation design is (almost) finished (including layout?) and they start with the next-next-gen design. And I'm also asking the same question. But more than a half a year before the CPU becomes available to the public it is already being tested by partners (mainboard manufacturers and ?).
Nvidia only for now. Precisely because they got fed up with Intel and AMD ( And Amphere as well ) for extremely slow PCIe adoption.
I believe only next Gen Intel and AMD Zen6 will get PCIe 6.0.
I am hoping Nvidia officially move into Server CPU market not only for their CPU but for wider Web Hosting as well. More competition for Server Hardware.
This actually makes sense from a spec perspective if you want to give enough to allow hardware to catch up with the specs and to support true interop.
Contrast this with the wild west that is "Ethernet" where it's extremely common for speeds to track well ahead of specs and where interop is, at best, "exciting."
I know very little about electronics design, so I always find it amazing that they keep managing to double PCIe throughput over and over. Its also probably the longest lived expansion bus at the moment.
Look at throughput on a single lane from the MAC to the PHY.
802.3dj is maybe finishing up soon and has 200gbps lanes. Which is more or less about what pci-e 8.x is supposed to be. The table in the article sums both directions of a lane, which leads to confusion. People want faster ethernet in fewer lanes, so no doubt a 400gbps per lane standard will be starting up soon for pci-e to leverage as pci-e 9.
> not in a format I would normally call Ethernet
Why, because it doesn't use ether, or vampire taps?
Ethernet works over many media, and that's been pretty consistent throughout its life. It started from or at least was inspired by AlohaNet a wireless system, then you had thick and thin coax, then twisted pair and fiber, now twinax and board level interfaces are common, too.
I'd highly advise against using GHz here (without further context, at least), a 32Gbaud / 32Gsym/s NRZ signal toggling at full rate is only a 16GHz square wave.
baud seems out of fashion, sym/s is pretty clear & unambiguous.
(And if you're talking channel bandwidth, that needs clarification)
> > I think Gen7 used 32GHz, which is ridiculously high.
> 16GHz square wave
Is it for PCIe 5.0? PCIe 6.0 should operate on the same frequency and doubling the bandwidth by using PAM4. If PCIe 7.0 doubled the bandwidth and is still PAM4, what is the underlying frequency?
And it's a good way to remove the ambiguity of things like DDR, but ugh "transfers" is not the best word here.
Looking at some documents from Micron I don't see them using GT/s anywhere. And in particular if I go look at their GDDR6X resources because those chips use PAM4, it's all about gigabits per second [per pin]. So for example 6GHz data clock, 12Gbaud, 24Gb/s/pin.
Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
PAM encoding is already analog, and also correspondingly more expensive (power, silicon size, etc) for the increase in speed.
It really wouldn't surprise me if even on workstation platforms only a subset of core lanes were Gen6+ and the common slots were redriven Gen5 or less off of a router / switch chip.
> Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
We don't have to go back, baud is still in use. I would expect transfers per second to be a synonym for baud though, and for bits per second per pin to use a different word.
Aside from GDDR for GPUs, DRAM is still mostly specified with MT/s rather than GT/s, probably because marketing prefers bigger numbers. It'll probably fall off once 5-digit numbers become commonplace.
A lot of people think that baud rate represents bits per second, which it only does in systems where the symbol set is binary. People got it from RS232.
From what I've seen, the faster PCI-E bus is important when you need to shuffle things in and out of VRAM. In a video game, the faster bus reduces the duration of stutters caused by pushing more data into the graphics card.
If you're using a new video card with only 8GB of onboard RAM and are turning on all the heavily-advertised bells and whistles on new games, you're going to be running out of VRAM very, very frequently. The faster bus isn't really important for higher frame rate, it makes the worst-case situations less bad.
I get the impression that many reviewers aren't equipped to do the sort of review that asks questions like "What's the intensity and frequency of the stuttering in the game?" because that's a bit harder than just looking at average, peak, and 90% frame rates. The question "How often do textures load at reduced resolution, or not at all?" probably requires a human in the loop to look at the rendered output to notice those sorts of errors... which is time consuming, attention-demanding work.
> I don't know how many games are even capable of using lower resolutions to avoid stutter.
Some games may be doing that. I expect that in others, the lower-resolution or missing textures are a result of the texture streaming system catastrophically failing to meet its deadline to load in the relevant textures and giving up. It's my understanding that "texture pop-in" is the too-late replacement of a low-resolution "placeholder" texture with a high-resolution texture. If the high-resolution texture doesn't load in time, then you're stuck with the low-res placeholder.
Commentary on textures that fail to load are in the "Monster Hunter: Wilds" section, starting at ~11:35 in [0], and the "Space Marine 2" section starting at ~00:14:40 in [1], which also mentions "Halo: Infinite" and "Forspoken" as other games that have the same sort of behavior. Missing textures are mentioned in the "Star Wars Jedi: Survivor" section starting at 21:21 at [1]. And -while not mentioned- if you look not-that-closely at the first ~5 seconds of that section, you can see the textures (most obviously the ground texture) go from "the same as the 16GB model" to "something you'd expect to see in a bad PS3 game".
Also in that first video, you can see some head-to-head demonstrations of the performance problems having a slower PCI-E link gives you when running out of VRAM starting at ~04:52 in the "The Last of Us Part II" section of [0] and also at ~17:00 in the "F1 25" section of that same video.
I expect there are a few other videos out there that do this sort of analysis, but I can't be arsed to find them.
No matter the leaps in bandwidth, the latency remains the same. Also, with PCIe switches used in AI servers, the latency (and jitter) is even pronounced.
what I don't get: why doesn't AMD just roll Gen6 out in their CPU, bifurcate it to Gen5, and boom, you have 48x2 Gen5s? same argument for gen5 bifurcated to gen4.
this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
Bifurcation can't create new lanes, only split the existing lane count up into separate logical slots.
What you're saying is possible though, you just need something a little heavier like a PCIe switch do the lane count + mixed-speed conversion magic. That's exactly what the chipset is, but for various reasons it's still only PCIe 4.0, even on the latest generation chips. I wouldn't be surprised if that changed again next generation. The downsides of this approach are switches add cost, latency, and can consume a lot of power. When they first upped the chipset to be a PCIe 4.0 connection in the 500 era, most motherboards capable of using the bandwidth of the chipset actually had to add chipset cooling.
Ideally they'd just add alternative options with more lanes directly from the CPU, but that'd add competition to the bottom of the Threadripper line.
> this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
Is that a problem?
These days, you don't need slots for your sound card and network card, that stuff's all integrated on the motherboard.
Plenty of enthusiast motherboards support 1 GPU, 3-4 nvme drives, 4 SATA drives, and have a PCIe 1x slot or two to spare.
Is anyone struggling, except folks working with LLMs? Seems to me folks looking to put several $2400 RTX 4090s in one machine ain't exactly a big market. And they'll probably want a giant server board, so they have space for all their giant four-slot-wide cards.
Devices integrated on the motherboard will either connect to the cpu via PCIe or USB, lanes aren't just for PCIe cards.
For example, motherboards with multiple nvme drives often have 1 drive with dedicated lanes and the remainder multiplexed through a PCIe switch embedded in the chipset.
I feel like what we really need is a GPU "socket" like we have for CPU's. And then a set of RAM slots dedicated to that GPU socket (or unified RAM shared between CPU and GPU)
Unfortunately socketed processors only really work with DDRx type DRAM interfaces. GPUs use GDDR and HBM interfaces, which are not ideal for sockets. In the case of HBM, you have 1024 data traces per DRAM chip, which would make the socket have an insane number of pins. GDDR has fewer pins, but makes up for it with higher data rates (32 pins at 16Gb/s) and that is impractical to use in a socket due to the variance in contact area resulting in impedance matching issues.
Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It seems like that would also have some power delivery advantages.
> at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory
Actually the RapsberryPi (appeared 2012) was based on a SoC with a big and powerful GPU and small weak supporting CPU. The board booted the GPU first.
If you look at a any of the nvidia DGX boards it's already pretty close.
PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...
There are companies that specialize in memory controller ip that every one else uses, including large semi companies like Intel.
The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.
These days you can buy any standard as a soft IP from Synopsys or Cadence. They take their previous serdes and modify it to meet the new standard. They have thousands of employees across the globe just doing that.
Isnt it about latency as well with DGX boards? Vs PCI-E. You can only fit so much RAM on a board that will realistically be plugging into a slot
Most current DGX server assemblies are stacked and compression-fit, much higher density and more amenable to liquid cooling.
https://www.servethehome.com/micron-socamm-memory-powers-nex...
Has the DGX actually shipped anywhere yet?
Do you mean the new one? The older ones have been around for so long you can buy off-leases of them: https://www.etb-tech.com/nvidia-dgx-1-ai-gpu-server-2-x-e5-2...
Good to see I’m not the only person that’s been thinking about this. Wedging gargantuan GPUs onto boards and into cases, sometimes needing support struts even, and pumping hundreds of watts through a power cable makes little sense to me. The CPU, RAM, these should be modules or cards on the GPU. Imagine that! CPU cards might be back..
It is not like CPU aren't getting higher wattage as well. Both AMD and Intel have roadmap for 800W CPU.
At 50-100W for IO, this only leaves 11W per Core on a 64 Core CPU.
800 watt CPU with a 600 watt GPU, I mean at a certain point people are going to need different wiring for outlets right?
This is a legitimate problem in datacenters. They're getting to the point where a single 40(ish)OU/RU rack can pull a megawatt in some hyperdense cases. The talk of GPU/AI datacenters consuming inordinate amounts of energy isn't just because the DC's are yuge, (although some are), but because the power draw per rack unit space is going through the roof as well.
On the consumer side of things where the CPU's are branded Ryzen or Core instead of Epyc or Xeon, a significant chunk of that power consumption is from the boosting behavior they implement to pseudo-artificially[0] inflate their performance numbers. You can save huge (easily 10%, often closer to 30%, but really depends on exact build/generation) on energy by doing a very mild undervolt and limiting boosting behavior on these cpus and keeping the same base clocks. Intel 11th through 14th gen CPU's are especially guilty of this, as are most Threadripper CPU's. you can often trade single digit or even negligible performance losses (depends on what you're using it for and how much you undervolt/underclock/restrict boosting) for double digit reductions in power usage. This phenomenon is also true for GPU's when compared across the enterprise/consumer divide, but not quite to the significant extent in most cases.
Point being, yeah, it's a problem in data centers, but honestly there's a lot of headroom still even if you only have your common American 15A@120VAC outlets available before you need to call your electrician and upgrade your panel and/or install 240VAC outlets or what have you.
0: I say pseudo-artificial because the performance advantages are real, but unless you're doing some intensive/extreme cooling, they aren't sustainable or indicative of nominal performance, just a brief bit of extra headroom before your cooling solution heat-soaks and the CPU/GPU's throttle themselves back down. But it lets them put the "Bigger number means better" on the box for marketing.
It's not just about better numbers. Getting high clocks for a short period helps in a lot of use cases - say random things like a search. If I'm looking for some specific phrase in my codebase in vscode, everything spins up for the second or two it takes to process that.
Boosting from 4 to 5,5.5 ghz for that brief period shaves a fraction of a second - repeat that for any similar operation and it adds up.
Yes, I figured that much would be obvious to this crowd. Thus the "pseudo" part.
The point isn't that there isn't a benefit, it's that you start to pay exponentially more energy per 0.1GHz at a certain point. Furthermore, AMD and Intel were exceptionally aggressive about it in the generations I outlined (AMD would be 7000 series ryzens specifically), leading to instability issues on both platforms due to their spec itself being too aggressive, or AIB partners improperly implementing that spec as the headroom that typically exists from factory stock to push clocks/voltages further was no longer there in some silicon (some of it comes down to silicon lottery and manufacturing defects/mistakes (Intel's oxidation issues for example) but we're really getting into the weeds on this already)
And to clarify: I'm talking specifically of Intel turboboost and AMD's PBO boosting technologies where they boost where they boost well over base clocks, separate from the general dynamic clocking behavior where clocks will drop well below base when not in (heavy) use.
unless it’s an Apple data center, populated by the server version of the latest ultra chips…
What makes you think that?
They're small and efficient, that means they can pack large numbers of those into small spaces, resulting in a similar large power draw per volume occupied by equipment in the DC. This is especially true with Apple's "Ultrafusion" tech which they're developing as quasi-analog to Nvidia Grace (Hopper) superchips.
Because I worked on them, before retiring. Yes they’re packed in; no they still don’t draw the same levels of power.
Didn't saw they draw the same, I openly acknowledge their more efficient. Said power user per rack unit is trending up. This is true of Apple DC's as well, especially with their new larger/fused chip initiatives. It's an universal industry trends especially with AI compute, and Apple is not immune.
Let me rephrase to: No, they (collectively) don’t draw the same levels of power. I know what amperage is drawn by each rack. It’s nowhere near as much as was drawn by the older intel-based racks.
And yes, they’re packed densely.
at that point, they're powered by a bicycle.
How safe is undervolting? Can it cause stability issues?
Far safer than overvolting.
Changing settings can lead to stability issues no matter which way you push it frankly. If you're don't know what you're doing/aren't comfortable with it, probably not worth it.
> They're getting to the point where a single 40(ish)OU/RU rack can pull a megawatt in some hyperdense cases.
Switch is designing for 2MW racks now.
At least with U.S. wiring we have 15 amps at 120 volts. For continuous power draw I know you'd want an 80% margin of safety, so let's say you have 1440 Watts of AC power you can safely draw continuously. Power supplies built on MOSFETs seem to peak at around 90% efficiency, but you could consider something like the Corsair AX1600i using gallium nitride transistors, which supposedly can handle up to 1600 watts at 94% efficiency.
Apparently we still have room, as long as you don't run anything else on the same circuit. :)
You can always have an electrician install a larger breaker for a particular circuit. I did that with my "server" area in my study, which was overkill cuz I barely pull 100w on it. But it cost nearly zero extra since he was doing a bunch of other things around the house anyway.
> You can always have an electrician install ...
If you own the house, sure. Many people don't.
You need to increase the wire diameter as well if you go that route. Running larger breakers on 10A or 15A wiring is a recipe for bad stuff.
In older houses, made from brick and concrete, that can be tricky to do. The only reason I can have my computer on a separate circuit is because we could repurpose the old three phase wiring for a sauna we ripped out. If that had not been the case, getting the wires to the fuse board would have been tricky at best.
New homes are probably worse than old homes through. The wires a just chucked in the space been the outer and inner walls, there's basically no chance of replacing them of pulling new ones. Old houses at least frequently have piping in which the wires run.
Larger breaker and thicker wires!
I thought you only needed thicker wires for higher amps? Should go without saying, but I am not a certified electrician :-)
I only have a PhD from YouTube (Electroboom)
The voltage is always going to be the same because the voltage is determined by the transformers leading to your service panel. The breakers break when you hit a certain amperage for a certain amount of time, so by installing a bigger breaker, you allow more amperage.
If you actually had an electrician do it, I doubt they would've installed a breaker if they thought the wiring wasn't sufficient. Truth is that you can indeed get away with a 20A circuit on 14 AWG wire if the run is short enough, though 12 AWG is recommended. The reason for this is voltage drop; the thinner gauge wire has more resistance, which causes more heat and voltage drop across the wire over the length of it, which can cause a fire if it gets sufficiently hot. I'm not sure how much risk you would put yourself in if you were out-of-spec a bit, but I wouldn't chance it personally.
Could you not just run a 240 volt outlet on existing wiring built for 110v? Just send l1 and l2 on the existing hot/neutral?
You can, 240V on normal 12/2 Romex is fine. The neutral needs to be "re-labeled" with tape at all junctions to signify that it's hot, and then this practice is (generally) even code compliant.
However! This strategy only works if the outlet was the only one on the circuit, and _that_ isn't particularly common.
Although this exists, as a layperson, I've rarely seen it. There is the NEMA 6-15R receptacle type, but I have literally none of those in my entire house, and I've really never seen them. Apparently they're sometimes used for air conditioners. Aside from the very common 5-15R, I see 5-20R (especially in businesses/hospitals), and 14-30R/14-50R for ranges and dryers. (I have one for my range, but here in the midwest electric dryers and ranges aren't as common, so you don't always come across these here. We have LNG ran to most properties.) So basically, I just really don't see a whole lot of NEMA 6 receptacles. The NEMA 14 receptacles, though, require both hots and the neutral, so in a typical U.S. service panel it requires a special breaker and to take up two slots, so definitely not as simple of a retrofit.
(Another outlet type I've seen: I saw a NEMA 7 277V receptacle before. I think you get this from one phase of a 480V three-phase system, which I understand is ran to many businesses.)
If you drive an electric car in a rural area you might want to carry around 6-30 and 6-50 adapters because most farms have welders plugged into those and that can give you a quick charge. And also TT-30 and 14-50 adapters to plug in at campgrounds.
NEMA 6 is limiting because there’s no neutral, so everything in the device has to run on 240V. Your oven and dryer want 120V to run lights and electronics, so they use a 14 (or 10 for older installs) which lets them get 120V between a hot and the neutral.
Oddly, 14-50 has become the most common receptacle for non-hardwired EV charging, which is rather wasteful since EV charging doesn’t need the neutral at all. 6-50 would make more sense there.
Reasons why it's nice to have a 14-50 plug in your garage rather than a 6-50:
1: when an uncle stops by for a visit with his RV he can plug in.
2: the other outlets in your garage are likely on a shared circuit. The 14-50 is dedicated, so with a 14-50 to 5-15 adapter you can more safely plug in a high wattage appliance, like a space heater.
1 is why we ended up with 14-50 as the standard, too. Before there was much charging infrastructure, RV parks were a good place to get a semi-fast charge, and that meant a charger with a 14-50 plug.
2 is something I never thought of, I’ll have to keep that in mind.
NEMA 6s are extremely common in barns and garages for welders. 6-50 is more common for bigger welders but I’ve also seen 6-20s on repurposed 12/2 Romex as the parent post was discussing used for cheap EV retrofits, compressors, and welders.
5-20R/6-20R is also somewhat commonly used by larger consumer UPS for your computer, router, etc.
Without upgrsding the wiring to a thicker gauge? That's not code compliant and is likely to cause a fire.
Sorry just to specify, it was more like a 20 amp I think (I will verify), it wasn't like I was going way higher.
I don't remember whether he ran another wire though. It was 5 years ago. Maybe I should not be spreading this anecdote without complete info.
He was a legit electrician that I've worked with for years, specifically because he doesn't cut corners. So I'm sure he did The Right Thing™.
If this is north america we're talking about, then 14 gauge is the standard for 120V 15A household circuits. By code, 20A requires 12 gauge. You'll notice the difference right away, it's noticeably harder to bend. Normally a house or condo will only have 15A wires running to circuits in the room. It's definitely not a standard upgrade, the 12 gauge wire costs a lot more per foot, no builder will do it unless the owner forks over extra dough.
Unless you performed the upgrade yourself or know for a fact that the wiring was upgraded to 12 gauge, it's very risky to just upgrade the breaker. That's how house fires start. It's worth it to check. If you know which breaker it is, you can see the gauge coming out. It's usually written on the wire.
I was actually under the impression that it is allowed depending on the length of the conductor, but it seems you are right. The NEC Table 15(B)(16) shows the maximum allowed ampacity of 14 AWG cables is 20 amperes, BUT... there is a footnote that states the following:
> * Unless otherwise specifically permitted elsewhere in this Code, the overcurrent protection for conductor types marked with an asterisk shall not exceed 15 amperes for No. 14 copper, 20 amperes for No. 12 copper, and 30 amperes for No. 10 copper, after any correction factors for ambient temperature and number of conductors have been applied.
I could've sworn there were actually some cases where it was allowed, but apparently not, or if there is, I'm not finding it. Seems like for 14 AWG cable the breaker can only be up to 15 amperes.
There is a chance he did not run new wires if he was able to ascertain that the wire gauge was sufficient to carry 20 amps over the length of the cable. This is a totally valid upgrade though it does obviously require you to be pretty sure you know the length of the entire circuit. If it was Southwire Romex, you can usually tell just by looking at the color of the sheathing on the cable (usually visible in the wallboxes.)
Where things get hairy are old houses with wiring that’s somewhere between shaky and a housefire waiting to happen, which are numerous.
As an old house owner, I can attest to that for sure. In fairness though, I suspect most of the atrocities occur in wall and work boxes, as long as your house is new enough to at least have NM sheathed wiring instead of ancient weird stuff like knob and tube. That's still bad but it's a solvable problem.
I've definitely seen my share of scary things. I have a lighting circuit that is incomprehensibly wired and seems to kill LED bulbs randomly during a power outage; I have zero clue what is going on with that one. Also, often times opening up wall boxes I will see backstabs that were not properly inserted or wire nuts that are just covering hand-twisted wires and not actually threaded at all (and not even the right size in some cases...) Needless to say, I should really get an electrician in here, but at least with a thermal camera you can look for signs of serious problems.
Yeah, but it ain't nothing that microwaves, space heaters, and hair dryers haven't already given a run for their money.
Hair dryers and microwaves only run for a few minutes, so even if you do have too much resistance this probably won't immediately reveal a problem. A space heater might, but most space heaters I've come across actually seem to draw not much over 1,000 watts.
And even then, even if you do run something 24/7 at max wattage, it's definitely not guaranteed to start a fire even if the wiring is bad. Like, as long as it's not egregiously bad, I'd expect that there's enough margin to cover up less severe issues in most cases. I'm guessing the most danger would come when it's particularly hot outside (especially since then you'll probably have a lot of heat exchangers running.)
That's still not much for wiring in most countries. A small IKEA consumer oven is only 230V16A=3860W. Those GPUs and CPUs only consume that much at max usage anyway. And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
> And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
Speak for yourself. I’d love to have that much computer at my disposal. Not sure what I’d do with it. Probably open Slack and Teams at the same time.
> Probably open Slack and Teams at the same time.
Too bad it feels like both might as well be single threaded applications somehow
I could use KVM and open a bunch of instances of each.
So Europe ends up with an incidental/accidental advantage in the AI race?
All American households get mains power at 240v (I'm missing some nuance here about poles and phases, so the electrical people can correct my terminology).
It's often used for things like ACs, Clothes Dryers, Stoves, EV Chargers.
So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
To get technical -- US homes get two phases of 120v that are 180 degrees out of phase with the neutral. Using either phase and the neutral gives you 120v. Using the two out of phase 120v phases together gives you a difference of 240v.
https://appliantology.org/uploads/monthly_2016_06/large.5758...
Even more technical, we don't have two phases, we have 1-phase that's split in half. I hate it cause it makes it confusing.
Two phase power is not the same as split phase (There's basically only weird older installations of 2 phase in use anymore).
Yeah that's right. The grid is three phases (as it is basically everywhere in the world), and the transformer at the pole splits one of those in half. Although, what are technically half-phases are usually just called "phases" when they're inside of a home.
Relevant video from Technology Connections:
"The US electrical system is not 120V" https://youtu.be/jMmUoZh3Hq4
That's such a great video, like most of his stuff.
> So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
It'd be all new wire run (120 is split at the panel, we aren't running 240v all over the house) and currently electricians are at a premium so it'd likely end up costing a thousand+ to run that if you're using an electrician, more if there's not clear access from an attic/basement/crawlspace.
Though I think it's unlikely we'll see an actual need for it at home, I imaging a 800w cpu is going to be for server class CPUs and rare-ish to see in home environments.
> and currently electricians are at a premium so it'd likely end up costing a thousand+
I got a quote for over 2 thousand to run a 24v line literally 9 feet from my electrical panel across my garage to put a EV charger in.
Opening up an actual wall and running it to another room? I can only imagine the insane quotes that'd get.
I kinda suspect there’s a premium once you mention “EV vehicle”, since you’re signalling that you’re affluent enough to afford an EV and have committed to spending the money required to get EV charging at home working, etc. (Kinda like getting a quote for anything wedding related.)
I’m getting some wiring run about the same distance (to my attic, fished up a wall, with moderately poor access) for non-EV purposes next week and the quote was a few hundred dollars.
[dead]
the trick is to request 240v outlet for welder. it brings price down to 400 or so.
running to another room will be done usually (at least in usa) through attic or crawlspace. i got it done a few months ago to have dedicated 20A circuit (for my rack) in my work room. cost was around 300-400 as well
Labor chargers alone are going to be higher than that in Seattle. Just to have someone come out on a call is going to be 150-200. If it is an independent electrician who owns their own business, maybe 100-150/hr, if they are part of a larger company, I'd expect even more than that.
Honestly I wouldn't expect to pay less than $1000 for the job w/o any markups.
i live in bay area. i have some doubts that seattle going to be more expensive.
Handy man prices around here are $65 to $100/hr, and there is a huge wait list for the good ones.
I've gotten multiple quotes on running the 240v line, the labor breakdown was always over $400 alone. Just having someone show up to do a job is going to be almost $200 before any work is done.
When I got quotes from unlicensed people, those came in around $1000 even.
in bayarea subreddits there are multiple posts talking about EV charger vs welder outlet and how it drops price from 2000 to 500 or so (depends on complexity).
another thing, which is good long term is to a find a local electrician (plumber, etc) who doesn't charge service calls and have reasonable pricing.
no idea about handyman pricing. never used any. for electrical/water/roofing i prefer somebody who is licensed/insured/bonded/etc
I don't think many people would want some 2kW+ system sitting on their desk at home anyways. That's quite a space heater to sit next to.
I should look at the label (or check with a meter..), but when I run my SGI Octane with its additional XIO SCSI board in active use, the little "office" room gets very hot indeed.
Also the noise from the fans.
If we're counting all the phases then european homes get 400V 3-phase, not 240V split-phase. Not that typical residential connections matter to highend servers.
It depends on the country, in many places independent houses get a single 230V phase only.
Well yes its possible but often $500-1000 to run a new 240v outlet, and that's to a garage for an ev charger. If you want an outlet in the house I dont know how much wall people want to tear up and extra time and cost.
Sure yeah, I was just clarifying that if the issue is 240v, etc, US houses have the feed coming in. Infrastructure-wise it's not an issue at all.
Consumers with desktop computers are not winning any AI race anywhere.
In residential power delivery? yes
In power cost? no
I'm literally any other way? also no
In the Nordics we're on 10A for standard wall outlets so we're stuck on 2300W without rewiring (or verifying wiring) to 2.5mm2.
We rarely use 16A but it exists. All buildings are connected to three phases so we can get the real juice when needed (apartments are often single phase).
I'm confident personal computers won't reach 2300W anytime soon though
In Italy we also have 10A and 16A (single phase). In practice however almost all wires running in the walls are 2.5 mm^2, so that you can use them for either one 16A plug or two adjacent 10A plugs.
In the Nordics (I'm assuming you mean Nordic countries) 10A is _not_ standard. Used to be, some forty years ago. Since then 16A is standard. My house has a few 10A leftovers from when the house was built, and after the change to TN which happened a couple of decades ago, and with new "modern" breakers, a single microwave oven on a 10A circuit is enough to trip the breaker (when the microwave pulses). Had to get the breakers changed to slow ones, but even those can get tripped by a microwave oven if there's something else (say, a kettle) on the same circuit.
16A is fine, for most things. 10A used to be kind of ok, with the old IT net and old-style fuses. Nowadays anything under 16A is useless for actual appliances. For the rest it's either 25A and a different plug, or 400V.
Let's rephrase: 10A is the effective standard that's been in use for a long long time, if you walk into a building you can assume it has 10A breakers.
On new installations you can choose 10A or 16A so if you're forward thinking you'd go 16 since it gives you another 1300 watts to play with.
[dead]
There already are different outlets for these higher power draw beasts in data centers. The amount of energy used in a 4u "AI" box is what an entire rack used to draw. Data centers themselves are having to rework/rewire areas in order to support these higher power systems.
You can up the voltage to 240 and re-use the wiring (with some minor mods to the ends), for double the power. Insulation class should be sufficient. That makes good sense anyway. You may still have an issue if the powersupply can't handle 240/60 but for most of the ones that I've used that would have worked. Better check with the manufacturer to be sure though. It's a lot easier and faster than rewiring.
A simple kitchen top water cooker is 2000W, so a 1500W PC sounds like no big deal.
Kettles in the US are usually 1500W, as the smallest branch circuits in US homes support 15A at 120V and the general rule for continuous loads is to be 80% of the maximum.
Ah, 16A at 230v (3680W) is a normal circuit here. Most appliances work with that, the common exception is electric cooking (using two circuits or 380v two-phase) and EV charging.
True but kettles rarely run for very long.
But computers do, which was why I included that context. You don't really want to build consumer PC >1500W in the US or you'd need to start changing the plug to patterns that require larger branch circuits.
Kettles and microwaves are usually 1100 watts and lower, but space heaters and car chargers can be 1500 watts and run for long periods of time.
Microwave ovens have a different issue, which I found when I upgraded my breaker board to a modern one in my house. The startup pulse gives a type of load which trips a standard A-type 10A breaker (230V). Had to get those changed to a "slow" type, but even that will trip every blue moon, and if there's something else significant on the same circuit the microwave oven will trip even so, every two weeks or so (for the record, I have several different types of microwave ovens around the house, and this happens everywhere there's a 10A circuit).
The newer circuits in the house are all 16A, but the old ones (very old) are 10A. A real pain, with new TN nets and modern breakers.
Microwave ovens top out around 1100-1250W output from a ~1500W input from the wall. Apparently there's a fair bit of energy lost in the power supply and magnetron that doesn't make it into the box where the food is.
You don't keep the kettle constantly running, unlike a PC.
And cooling. Look here: https://www.fz-juelich.de/en/news/archive/press-release/2025...
Especially a special PDU: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202...
And cooling: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202...
It is mostly an issue in countries with 120V mains (I know that in the US 240V outlets exist though). In France for example it is required that standard plugs must be able to deliver at least 16A on each outlet, at the 230V used here, we get 3600W of power, that’s more than enough.
Laughs in 230V (sorry).
ʰᵉₕₑheʰᵉₕₑhe in 400V
Yes and this is something I've been thinking about for awhile.
A computer is becoming a Home Appliance in the it will need 20A wiring and plugs soon, but should move to 220/240v soon anyway (and change the jumper on your standard power supply).
But all of the most-ridiculous hyperscale deployments, where bandwidth + latency most matter, have multiple GPUs per CPU, with the CPU responsible for splitting/packing/scheduling models and inference workloads across its own direct-attached GPUs, providing the network the abstraction of a single GPU with more (NUMA) VRAM than is possible for any single physical GPU to have.
How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P
GPU as motherboard really only makes sense for gaming PCs. Even there SXM might be easier.
No, for a gaming computer what we need is the motherboard and gpu to be side by side. That way the heat sinks for the CPU and GPU have similar amounts of space available.
For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.
https://en.wikipedia.org/wiki/Compute_Express_Link
It’s always going to be a back and forth on how you attach stuff.
Maybe the GPU becomes the motherboard and the CPU plugs into it.
And the memory should be a onboard module on the cpu card intel/amd should replicate what apple did with a unified same ringbus sort of memory modules. Lower latency,higher throughput.
Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.
That would pretty much make both intel and amd to start market segmentation by CPU Core + Memory combination. I absolutely do not want that.
We're already there. That's what a lot of people are using DPU's are for.
An example, This is storage instead of GPU's, but as the SSD's were PCIe NVMe, it's pretty nearly the same concept: https://www.servethehome.com/zfs-without-a-server-using-the-...
To continue the ServeAtHome links, https://www.servethehome.com/microchip-adaptec-smartraid-430...
PCI-e Networks and CXL are the future of many platforms... like ISA backplanes.
Yep, I have a lot of experience with CXL devices and networked PCIe/NVMe (over Eth/IB) Fabrics and deploying "Headless"/"Micro-Head" compute units which are essentially just a pair of DPU's on a PCIe multiplexer (basically just a bunch of PCIe slots tied to a PCIe Switch or two).
That said my experience in this field is more with storage than GPU compute, but I have done some limited hacking about in the GPGPU space with that tech as well. Really fascinating stuff (and often hard to keep up with and making sure every part in the chain supports the features you want to leverage, not to mention going down the PCIe root topology rabbit hole and dealing with latency/trace-length/SnR issues with retimers vs muxers vs etc etc etc).
It's still a nascent field that's very expensive to play in, but I agree it's the future of at least part of the data infrastructure field.
Really looking forward to finally getting my hands on CXL3.x stuff (outside of a demo environment.)
EE here. There's no reason to not deliver power directly to the GPU by using cables. I'm not sure if it's sooving anything.
But you are right, there's no hierarchy in the systems anymore. Why do we even call something a motherboard? There's a bunch of chips interconnected.
Can I just have a backplane? Pretty please?
I've wondered why there hasn't been a desktop with a CPU+RAM card that slots into a PCIe x32 slot (if such a thing could exist), or maybe dual x16 slots, and the motherboard could be a dumb backplane that only connected the other slots and distributed power, and probably be much smaller.
Those exist; they are used for risers ("vertical mount GPU brackets, for dual GPU" equivalent for servers, where they make the cards flat again).
PCIe x 32 actually exists, at least in the specification. I have never seen a picture of a part using it.
Retimers.
Sockets (and especially backplanes) are absolutely atrocious for signal integrity.
I guess if it's possible to have 30cm PCIe 5 riser cables, it should be possible to have a backplane with traces of similar length.
Cables much better sadly, so much so that they started to use cables to jump across the server main board in places.
VMEBus for the win! (now VPX...)
The hot stuff nowadays is µTCA: https://www.picmg.org/openstandards/microtca/
If I remember correctly the military / aerospace shy away from this spec because the connector with the pins is on the backplane, with the sockets on the cards.
So if you incorrectly insert a card and bend a pin you're in trouble.
VPX has the sockets on the backplane so avoids this issue, if you bend pins you just grab another card from spares.
This may have changed since I last looked at it.
Telecoms industry definitely seem to favour TCA though.
I don't know, I work in particle physics and here µTCA is all the rage nowadays.
Yes, for fucks sake, this is the only way forward. It gives us the ultimate freedom to do whatever we want in the future. Just make everything a card on the bus and quit with all this hierarchy nonsense.
Wouldn't that mean an complete mobo replacement to upgrade the GPU? GPU upgrades seem much more rapid and substantial compared to CPU/RAM. Each upgrade would now mean taking out the CPU/RAM and other cards vs just replacing the GPU
GPUs completely dominate the cost of a server, so a GPU upgrade typically means new servers.
Agree - newer GPU likely will need faster PCIe speeds too.
Kinda like RAM - almost useless in terms of “upgrade” if one waits a few years. (Seems like DDR4 didn’t last long!)
> GPU upgrades seem much more rapid and substantial compared to CPU/RAM.
I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.
Yes I agree, let's bring back the SECC style CPU's from the Pentium Era, I've still got my Pentium II (with MMX technology)
And limit yourself to only one GPU?
Also CPUs are able to make use of more space for memory, both horizontally and vertically.
I don't really see the power delivery advantages, either way you're running a bunch of EPS12V or similar cables around.
Personally I hope this point comes after we realise we don't need 1kW GPUs doing a whole lot of not much useful
Figure out how much RAM, L1-3|4 cache, integer, vector, graphics, and AI horsepower is needed for a use-case ahead-of-time and cram them all into one huge socket with intensive power rails and cooling. The internal RAM bus doesn't have to be DDRn/X either. An integrated northbridge would deliver PCIe, etc.
I wonder how many additional layers are required in the PCB to achieve this + how this will dramatically affect the TDP; the GPU's aren't the only components with heat tolerance and capacitance.
It is not a EE problem. It is an ecosystem problem. You need a whole catalog of compatible hardware for this.
The concept exists now. You can "reverse offload" work to the CPU.
Isn't that what has kinda sorta basically happened with Apple Silicon?
And AMD Strix Halo.
GPU + CPU on the same die, RAM on the same package.
A total computer all-in-one. Just no interface to the world without the motherboard.
One possible advantage of this approach that no one here has mentioned yet is that it would allow us to put RAM on the CPU die (allowing for us to take advantage of the greater memory bandwidth) while also allowing for upgradable RAM.
I think you'd want to go the other way.
GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds.
It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory?
It's a whole new architecture but it seems to get at the actual problems we have in the space.
Isn't that what you described Direct Storage?
You're still running through the PCIe slot and it's bandwidth limit. I'm suggesting you bypass even that and put more memory directly on the card.
So an additional layer slower and larger than global GPU memory?
I believe that's kind of what bolt graphics is doing with the dimm slots next to the soldered on lpddr5. https://bolt.graphics/how-it-works/
Couldn’t we do that today if we wanted to?
What’s keeping Intel/AMD from putting memory on package like Apple does other than cost and possibly consumer demand?
Supply + demand, the manufacturing-capacity rabbit hole.
Bring back the S100 bus and put literally everything on a card. Your motherboard is just a dumb bus backplane.
We were moving that way, sorta, with Slot 1 and Slot A.
Then that became unnecessary when L2 cache went on-die.
I love the PCIe standard is 3 generations ahead of what is actually released. Gen5 is the live version, but the team behind it is so well organized that they have a roadmap of 3 additional versions now. Love it.
"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today, PCIe 7.x was finalized last month, and this is an announcement work on PCIe 8.0 has started for release in 3 years. I.e. it has only been one month of being one generation behind the latest PCIe revision.
It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.
either I'm misinformed in which case is like to see proof, or you are, but GB200 is not Gen6, just Gen5 and this will only be rectified with GB300 which, while exists in the wild, is not in the millions.
so indeed the parent commenter would be correct that everything is Gen5 right now.
I'm going off of https://developer.nvidia.com/blog/nvidia-gb200-nvl72-deliver... but I have not actually deployed a GB200 system yet myself. Consumer Blackwell is definitely 5.0 though.
that article is from last March. the plan was indeed for the GB200 board to support gen6 but it didn't happen and GB300 will rectify it.
at least that's my best educated guess, looking at supermicro's public spec sheet and that it's shipping with CX7 which is Gen5 and not with CX8.
B200 supports Gen6 there is just nothing that would let it run at Gen6.
At least the GB200s OpenAI purchased have CX-8s, but it's very possible (perhaps even very likely) most of the millions of GB200s deployed PCIe 5.0 using CX-7s given the SuperMicro spec sheet.
Edit: found this SemiAnalysis post saying as much https://x.com/SemiAnalysis_/status/1947768988467138645/photo...
thanks! very interesting that they got a custom SKU.
Datacenter Blackwell is Gen6, which is critical when pairing it with CX8 (2x400G) as otherwise you'd be stranding NIC BW.
please link me a B200 (not a B300) system with CX8 that uses gen6 x16 instead of gen5 x32.
to the best of my knowledge this does not exist, but I'd be happy to stand corrected.
(the official NVIDIA DGX B200 is gen5).
CPUs themselves only have so many PCI-E lanes though right? Wouldnt it make sense (even for consumers) to have peripherals using less lanes (but more speed per lane) for a multi GPU system or something that uses a lot of drives?
More lanes = more cost
Faster lanes = more cost
More faster lanes = lots more cost
The chipset also strikes some of the balance for consumers though. It has a narrow high speed connection to the CPU but enables many lower speed devices to share that bandwidth. That way you can have your spare NVMe drive, SATA controller, wired and wireless NICs, sound hardware, most of your USB ports, your capture card, and some other random things connected over a single x4 to x8 sized channel. This leaves the high cost lanes for just the devices that actually use them (GPU and primary, possibly secondary, storage drive). I've got one consumer type Motherboard with 14 NVMe drives connected, for example, just not at full native speed directly to the CPU.
You're just SoL if you want to connect a bunch of really high bandwidth devices simultaneously (100 Gbps+ NICs, multiple GPUs at full connection speed, a dozen NVMe drives at native speed, or similar) because then you'll be paying for a workstation/server class platform which did make the "more faster lanes" tradeoff (plus some market segment gouging, of course).
One issue is that, at least on cheaper mobos, these don't work as a "total bandwidth budget" situation. And, especially with newer generation PCIe, it can be a bit frustrating.
Many mobos will operate the available slots such that the total number of active lanes is split between them. But if you use older-generation cards, you'll only get a fraction of the available bandwidth because you're only using a fraction of their lanes, although the physical lanes are physically present.
What I'm thinking about is something like, say, a pair of Gen3 NVMe drives that are good enough for mass storage (running in RAID-1 for good measure) and some cheap used 10 Gb NIC, which will probably be 8x Gen2, all running on a Gen4+ capable mobo.
And, while for a general-purpose setup I can live with splitting available BW between the NIC and the GPU (I most likely don't care about my download going super fast while I game), the downloads will generally go to the storage, so they must be fast at the same time.
Those MBs are cheaper precisely because supporting this kind of bandwidth breakout adds cost (-> a fancier PCIe switch in a higher end chipset/southbridge). If you add the support to do this to them you end up with the more expensive motherboard. Some of the highest end motherboards actually have 2 chipsets/PCIe switches, more cost but a more bandwidth sharing for the same number of lanes coming from the CPU.
You can also buy external PCIe switches (just make sure you're not accidentally buying a PCIe bifurcation device). Most of the time it's cheaper to just buy the higher end motherboard though, e.g. I don't want to know what price "Request a quote" for this PCIe switch which can do x8 4.0 upstream and then quad 4x 3.0 downstream https://www.amfeltec.com/pci-express-gen-4-carrier-board-for... is. I do have a few 3.0 era cards which were more reasonably priced though https://www.aliexpress.us/item/3256801702762036.html?gateway... and they've worked well for me.
How high-end are we talking about? Do you know off-hand of any models supporting this?
I haven't seen such features on boards under 200 EUR, from Asus, Asrock and Gigabyte.
The thing is, if I have to splurge for some 400 EUR "gaming" model, I might as well move to a "workstation" CPU supporting more lanes out of the box, and the mobo will be priced roughly the same.
> "3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today
True. And yet, if you buy an RTX 5090 today, costing $2400 and released in January this year, it's PCIe 5.0 x16
Does the second half not already talk to this? Adding on to it further, GN found a few % performance difference on a 5090 going from PCIe 3.0 to 5.0 https://gamersnexus.net/gpus/nvidia-rtx-5090-pcie-50-vs-40-v... meanwhile many DC use cases find PCIe 5.0 unusably slow, driving alternative (non-PCIe) interconnects.
It takes a long time to get form standard to silicon, so I bet there are design teams working on pcie7 right now, which won't see products for 2 or more years
Is there an advantage of getting so far ahead of implementations? It seems like it would be more difficult to incorporate lessons.
When AMD introduces a new Desktop CPU series IIRC they claim the next generation design is (almost) finished (including layout?) and they start with the next-next-gen design. And I'm also asking the same question. But more than a half a year before the CPU becomes available to the public it is already being tested by partners (mainboard manufacturers and ?).
Gen6 is in use look at Nvidia ConnectX-8
What hosts support Gen6? AFAIK, Gen5 is the most recent standard that's actually deployed. Eg, what can you plug a CX8 into that will link up at Gen6?
into a Gen6 Bianca board (baseboard for GB300). that's it today.
how you use those today is twofold:
- Gen5 x32 via two x16 slots (this is how most people use them)
- use actually the CX8 as your PCIe switch directly to your GPUs: https://www.servethehome.com/this-is-the-nvidia-mgx-pcie-swi...
Nvidia only for now. Precisely because they got fed up with Intel and AMD ( And Amphere as well ) for extremely slow PCIe adoption.
I believe only next Gen Intel and AMD Zen6 will get PCIe 6.0.
I am hoping Nvidia officially move into Server CPU market not only for their CPU but for wider Web Hosting as well. More competition for Server Hardware.
Blackwell DC (B200/B300)
while the B200 chip itself could do PCIe6 as it was planned for GB200, there is no system around it with Gen6. the official DGX B200 is just PCIe5.
GB300 is indeed Gen6.
Custom Nvidia network cards I guess.
This actually makes sense from a spec perspective if you want to give enough to allow hardware to catch up with the specs and to support true interop.
Contrast this with the wild west that is "Ethernet" where it's extremely common for speeds to track well ahead of specs and where interop is, at best, "exciting."
So we can skip 6 and 7 and go directly to 8, right?
I know very little about electronics design, so I always find it amazing that they keep managing to double PCIe throughput over and over. Its also probably the longest lived expansion bus at the moment.
It's less surprising if you realize that PCIe is behind Ethernet (per lane).
Which Ethernet standard? Light online searching leads me to believe it tops out at 800Gb/s. (and not in a format I would normally call Ethernet)
Look at throughput on a single lane from the MAC to the PHY.
802.3dj is maybe finishing up soon and has 200gbps lanes. Which is more or less about what pci-e 8.x is supposed to be. The table in the article sums both directions of a lane, which leads to confusion. People want faster ethernet in fewer lanes, so no doubt a 400gbps per lane standard will be starting up soon for pci-e to leverage as pci-e 9.
> not in a format I would normally call Ethernet
Why, because it doesn't use ether, or vampire taps?
Ethernet works over many media, and that's been pretty consistent throughout its life. It started from or at least was inspired by AlohaNet a wireless system, then you had thick and thin coax, then twisted pair and fiber, now twinax and board level interfaces are common, too.
Ethernet has 100 Gbps per lane today and soon will have 200 Gbps while PCIe has 32 GBps today and soon will have 64 Gbps per lane.
I’m sure you can get some VMEbus boards.
My reaction to PCIe gen 8 is essentially "Huh? No, retro data buses are like ISA, PCI, and AGP, right? PCIe Gen 3 and SATA are still pretty new...".
I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
> PCIe Gen 3 and SATA are still pretty new...
That's an interesting thought to look at. PCIe 3 was a while ago, but SATA was nearly a decade before that.
> I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
Wikipedia says it's planned to be PAM4 just like 6 and 7.
Gen 5 and 6 were 32 gigabaud. If 8 is PAM4 it'll be 128 gigabaud...
I'd highly advise against using GHz here (without further context, at least), a 32Gbaud / 32Gsym/s NRZ signal toggling at full rate is only a 16GHz square wave.
baud seems out of fashion, sym/s is pretty clear & unambiguous.
(And if you're talking channel bandwidth, that needs clarification)
> > I think Gen7 used 32GHz, which is ridiculously high.
> 16GHz square wave
Is it for PCIe 5.0? PCIe 6.0 should operate on the same frequency and doubling the bandwidth by using PAM4. If PCIe 7.0 doubled the bandwidth and is still PAM4, what is the underlying frequency?
PCIe 7 = 128 GT/s = 64 Gbaud × PAM-4 = 32 "GHz" (if you alternate extremes on each symbol)
for gen6, halve all numbers
Is it me or are they using the term GigaTransfers wrong? They're counting a single PAM4 pulse as two "transfers".
They kinda are and kinda aren't, they're just using their own definition…
(I'm accepting it because "Transfers"/"T" as unit is quite rare outside of PCIe)
GT/s is also gaining ground for system RAM in order to clear up the ambiguity that DDR causes for end-consumers.
And it's a good way to remove the ambiguity of things like DDR, but ugh "transfers" is not the best word here.
Looking at some documents from Micron I don't see them using GT/s anywhere. And in particular if I go look at their GDDR6X resources because those chips use PAM4, it's all about gigabits per second [per pin]. So for example 6GHz data clock, 12Gbaud, 24Gb/s/pin.
Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
PAM encoding is already analog, and also correspondingly more expensive (power, silicon size, etc) for the increase in speed.
It really wouldn't surprise me if even on workstation platforms only a subset of core lanes were Gen6+ and the common slots were redriven Gen5 or less off of a router / switch chip.
> Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
We don't have to go back, baud is still in use. I would expect transfers per second to be a synonym for baud though, and for bits per second per pin to use a different word.
Aside from GDDR for GPUs, DRAM is still mostly specified with MT/s rather than GT/s, probably because marketing prefers bigger numbers. It'll probably fall off once 5-digit numbers become commonplace.
> baud seems out of fashion, sym/s is pretty clear & unambiguous.
Huh? Baud is sym/s.
Yes, that was the implication, but I've been getting the impression that using baud is kinda unpopular compared to using sym/s.
A lot of people think that baud rate represents bits per second, which it only does in systems where the symbol set is binary. People got it from RS232.
IIRC, modems never went much beyond 2400 baud. Everything past that was clever modulation packing more bits onto a single symbol.
Huh, I never thought of it that way but you're right.
Don't forget VESA Local Bus.
Meanwhile paying a premium for a Gen5 motherboard may net you somewhere in the realm of 4% improvements in gaming if you're lucky.
Obviously PCI is not just about gaming but...
From what I've seen, the faster PCI-E bus is important when you need to shuffle things in and out of VRAM. In a video game, the faster bus reduces the duration of stutters caused by pushing more data into the graphics card.
If you're using a new video card with only 8GB of onboard RAM and are turning on all the heavily-advertised bells and whistles on new games, you're going to be running out of VRAM very, very frequently. The faster bus isn't really important for higher frame rate, it makes the worst-case situations less bad.
I get the impression that many reviewers aren't equipped to do the sort of review that asks questions like "What's the intensity and frequency of the stuttering in the game?" because that's a bit harder than just looking at average, peak, and 90% frame rates. The question "How often do textures load at reduced resolution, or not at all?" probably requires a human in the loop to look at the rendered output to notice those sorts of errors... which is time consuming, attention-demanding work.
There's a good amount of reviewers showing 1% lows and 0.1% lows, which should capture stuttering pretty well.
I don't know how many games are even capable of using lower resolutions to avoid stutter. I'd be interested in an analysis.
> I don't know how many games are even capable of using lower resolutions to avoid stutter.
Some games may be doing that. I expect that in others, the lower-resolution or missing textures are a result of the texture streaming system catastrophically failing to meet its deadline to load in the relevant textures and giving up. It's my understanding that "texture pop-in" is the too-late replacement of a low-resolution "placeholder" texture with a high-resolution texture. If the high-resolution texture doesn't load in time, then you're stuck with the low-res placeholder.
Commentary on textures that fail to load are in the "Monster Hunter: Wilds" section, starting at ~11:35 in [0], and the "Space Marine 2" section starting at ~00:14:40 in [1], which also mentions "Halo: Infinite" and "Forspoken" as other games that have the same sort of behavior. Missing textures are mentioned in the "Star Wars Jedi: Survivor" section starting at 21:21 at [1]. And -while not mentioned- if you look not-that-closely at the first ~5 seconds of that section, you can see the textures (most obviously the ground texture) go from "the same as the 16GB model" to "something you'd expect to see in a bad PS3 game".
Also in that first video, you can see some head-to-head demonstrations of the performance problems having a slower PCI-E link gives you when running out of VRAM starting at ~04:52 in the "The Last of Us Part II" section of [0] and also at ~17:00 in the "F1 25" section of that same video.
I expect there are a few other videos out there that do this sort of analysis, but I can't be arsed to find them.
[0] <https://www.youtube.com/watch?v=MG9mFS7lMzU>
[1] <https://www.youtube.com/watch?v=AdZoa6Gzl6s>
I’m sure Windows performance counters can track the volume of data going between CPU memory and VRAM over the PCIe bus.
No matter the leaps in bandwidth, the latency remains the same. Also, with PCIe switches used in AI servers, the latency (and jitter) is even pronounced.
By an overwhelming margin, most computers are not in gamers' basements.
I thought we were only just up to 5? Did we skip 6 and 7?
Some of the newer ones maybe more for data centers.
Yeah, such a shame I've just upgraded to a 7.0 motherboard for my socket AM7 CPU.
Being less sarcastic, I would ask if 6.0 mobos are on the horizon.
I guess Venice, Diamond Rapids, and Vera will have 6.0.
GB300 has Gen6 so I am pretty sure GV200 would have it at least.
I'm fairly sure they are cooking Gen7 already into CX9.
I wonder if this will help applications like VPP/DPDK. not sure if the CPU or the lanes are the bottleneck there.
For all the others seeing the HN Hugg of death :)
https://archive.is/oa81K
Crazy to see STH get the HN hug of death!
One would think they get sizeable traffic as-is.
I can't be the only one let down that there wasn't some new slot design. Something with pizzazz and flare.
I'll take it if my consumer mb chipset supports giving me 48 PCIe7 lanes if future desktops still would only come with 24 gen 8 lanes
So I can run a full powered PCIe 4.0 16x GPU on 1 PCIe 8.0 lane???
what I don't get: why doesn't AMD just roll Gen6 out in their CPU, bifurcate it to Gen5, and boom, you have 48x2 Gen5s? same argument for gen5 bifurcated to gen4.
this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
Bifurcation can't create new lanes, only split the existing lane count up into separate logical slots.
What you're saying is possible though, you just need something a little heavier like a PCIe switch do the lane count + mixed-speed conversion magic. That's exactly what the chipset is, but for various reasons it's still only PCIe 4.0, even on the latest generation chips. I wouldn't be surprised if that changed again next generation. The downsides of this approach are switches add cost, latency, and can consume a lot of power. When they first upped the chipset to be a PCIe 4.0 connection in the 500 era, most motherboards capable of using the bandwidth of the chipset actually had to add chipset cooling.
Ideally they'd just add alternative options with more lanes directly from the CPU, but that'd add competition to the bottom of the Threadripper line.
> this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
Is that a problem?
These days, you don't need slots for your sound card and network card, that stuff's all integrated on the motherboard.
Plenty of enthusiast motherboards support 1 GPU, 3-4 nvme drives, 4 SATA drives, and have a PCIe 1x slot or two to spare.
Is anyone struggling, except folks working with LLMs? Seems to me folks looking to put several $2400 RTX 4090s in one machine ain't exactly a big market. And they'll probably want a giant server board, so they have space for all their giant four-slot-wide cards.
Devices integrated on the motherboard will either connect to the cpu via PCIe or USB, lanes aren't just for PCIe cards.
For example, motherboards with multiple nvme drives often have 1 drive with dedicated lanes and the remainder multiplexed through a PCIe switch embedded in the chipset.
I feel like what we really need is a GPU "socket" like we have for CPU's. And then a set of RAM slots dedicated to that GPU socket (or unified RAM shared between CPU and GPU)
Unfortunately socketed processors only really work with DDRx type DRAM interfaces. GPUs use GDDR and HBM interfaces, which are not ideal for sockets. In the case of HBM, you have 1024 data traces per DRAM chip, which would make the socket have an insane number of pins. GDDR has fewer pins, but makes up for it with higher data rates (32 pins at 16Gb/s) and that is impractical to use in a socket due to the variance in contact area resulting in impedance matching issues.
HBM would most likely have to be integrated directly on the GPU module given performance demands and signal constraints.
[dead]
[dead]