H100 vs H200 vs B200 — Specs, Memory and Buying Used

Quick answer: The H100 and H200 run the same GH100 die pair at the same clocks — the only difference is memory, 80 GB of HBM3 at 3.35 TB/s versus 141 GB of HBM3e at 4.8 TB/s. B200 is a new generation: two dies per package, 180 GB of HBM3e at 8 TB/s, roughly double the dense FP16 compute, NVLink 5, and about 1 kW per card. If your models fit in 80 GB per GPU, an H100 SXM is the simplest buy. If weights plus KV cache need more than that, the H200 saves you a second card per replica. If you need Blackwell-class compute density, budget for liquid cooling before you budget for anything else.

This guide deliberately contains no prices. Accelerator quotes move week to week with condition grade, memory configuration, batch size and destination, so any number printed here would be wrong by the time you read it. What does not move is the specification and the procurement logic — which is what decides whether a quote is good.

The spec sheet

Vendor figures, dense (sparsity off).

Two rows are easy to overlook. The H100 PCIe is not a slower H100 SXM — it is a different memory subsystem (HBM2e at 2 TB/s) and a different fabric story: two-way only, 600 GB/s, with no path to an eight-card all-to-all mesh. And the L40S has no card-to-card interconnect at all, which is fine for inference and rendering and disqualifying for anything that needs tensor parallelism across eight GPUs.

If you are specifying a PCIe platform, note that NVL variants exist: H100 NVL at 94 GB (3.9 TB/s, up to 400 W) and H200 NVL at 141 GB (600 W). Both are PCIe cards bridged in pairs, not in an eight-way mesh.

The ratios that decide the fit

Prices change; these ratios do not. They are computed from the table above, so you can check any quote against them.

Read with your bottleneck in mind, because the winners are different.

• Capacity-bound work — large models, long context, many concurrent sequences — is decided by HBM per card and by TFLOPS per GB of HBM. The low end of that last column is the point: MI300X and H200 carry the most memory for the compute they bring, which is exactly what you want when the GPU is mostly holding weights and KV cache.
• Bandwidth-bound work — token generation on any transformer — is decided by bandwidth per GB of HBM. H100 SXM and B200 lead here, which is why a 3.35 TB/s H100 SXM usually beats a 2.0 TB/s H100 PCIe at the same model size, despite identical model names.
• Power- or rack-constrained work is decided by TFLOPS per watt and by bandwidth per watt. L40S and A100 sit at the bottom; Blackwell sits at the top and costs you cooling to get there.

And the one column that is not in the table: fabric. A card with no NVLink is not a cheaper version of a card with NVLink, it is a different platform, and no ratio will tell you that.

H100 vs H200: same compute, different memory

Strip the labels and the H200 is an H100 SXM with 61 GB more HBM3e and 43% more bandwidth. Same GH100 die pair, same 989 TFLOPS dense, same 900 GB/s NVLink, same 700 W. Everything that changes is memory.

That is exactly the axis that governs inference economics in 2026. Token generation is memory-bandwidth-bound: every token requires reading the full weight set plus the KV cache, so tokens per second scale roughly with bandwidth per GPU, and concurrency scales with capacity. Going from 3.35 TB/s to 4.8 TB/s is a 43% headroom increase on the number that sets your serving throughput — before you count the fact that you can hold a larger KV cache for the same model on the same card.

Where the H200's extra memory does not earn its place: models under ~20B parameters with short context (both cards fit them several times over), fine-tuning jobs where step time is compute-bound rather than bandwidth-bound, and anything where you can shard across more nodes cheaply. In those cases an H100 SXM build with more cards usually beats a smaller H200 build — more cards buy more parallelism, and parallelism is what most training and batch workloads are actually short of.

The 70B test

For a 70B parameter model at FP16, the weights alone are about 140 GB. One H200 holds them with room for KV cache. One H100 80 GB does not — that design needs two cards per replica, or FP8 quantisation, or model sharding with the associated interconnect traffic.

Two H100 SXM cards also mean 1,400 W of accelerator power and two failure surfaces for one replica, against 700 W and one card for the H200. That is the real comparison: it is not "80 GB versus 141 GB", it is "one card versus two, in the same chassis budget, on the same baseboard generation". Run the same calculation for your own model size before you pick, because it flips at different points for 7B, 20B and 70B weights.

B200: double the compute, and a power problem

B200 is the first Blackwell part most buyers will actually quote. Two dies in one package, 180 GB HBM3e at 8 TB/s, ~2.25 PFLOPS dense FP16, NVLink 5 at 1.8 TB/s, around 1 kW per card. It is roughly double the H100's compute and more than double its bandwidth, and it is the only card in this table where the compute increase outruns the memory increase.

The constraint moves with it. Eight B200 cards draw about 8 kW before CPUs, NICs and drives; a full node lands at 10–12 kW, and eight such nodes put 80–96 kW in a rack. Air-cooled density tops out well below that, so a B200 decision is a facilities decision: liquid cooling, power provisioning at the colo and per-rack limits become the binding constraint before the budget does. H100 and H200 nodes sit at roughly 7 kW each, which is why so many 2026 builds still specify Hopper — not because it is cheaper per FLOP, but because the datacenter can cool it.

Supply is the other half. B200 is largely allocated through OEM and distributor channels; genuine secondary supply is thin and should be treated with suspicion. Lead times of 12–20 weeks are normal, and an allocation slot is not the same thing as stock on a shelf. Blackwell Ultra (B300, 288 GB at 1.4 kW) is the next planning point, and it lands in the same power envelope problem.

What an 8-card node actually contains

The most common sourcing mistake is quoting cards and forgetting the platform. An 8-GPU SXM node is a matched set, and the parts are generation-specific.

Two procurement notes follow. First, quote the node, not the cards — an SXM5 module is not a PCIe card, it needs an HGX baseboard, a matching power shelf and a chassis with the right thermal design, and those parts are not interchangeable between H100 and B200 generations. Second, treat memory as an accelerant of the schedule, not an afterthought: server DRAM has been volatile through 2026, as our DDR4 vs DDR5 bulk cost breakdown explains, and 32 DIMMs per node is not a rounding error — that line is priced off the server memory line and it moves weekly.

The composition of an 8-GPU node is stable even though the numbers are not: accelerators dominate the build, memory is the second-largest line item in a way it was not in 2023, and the CPU that people argue about most is the smallest of the three.

Which card for which workload

Every card in this table is quoted from held stock in Hong Kong, and current batches are listed on the data-centre GPU line.

What the used market gets wrong

Three failure modes account for most of the disputes we see in bulk GPU orders, and all three are detectable before payment.

• Hours that do not exist. nvidia-smi does not report GPU-on hours. A supplier who quotes them "from nvidia-smi" has invented them; the real sources are donor-chassis BMC power-on logs and the OEM warranty start date, which should corroborate each other.
• SXM modules sold as standalone cards. Hopper SXM5 parts are baseboard components. If a quote offers "H100 SXM, works in any server", the unstated part of the deal is that you also need an HGX baseboard, power shelf and chassis — or that the module came off one and has an unknown interconnect history.
• Cosmetic refurbishment as health. Fresh thermal pads and paste make a card look serviced and change its thermal signature. Sustained two-to-four-hour load runs, with the clock and thermal curve logged, are what separate maintenance from concealment.

The verification routine covers the countermeasures in full: aggregate ECC counters, row-remap and page-retirement state, serial lists in the PO, and NCCL bandwidth against a reference group.

Notes from the Zhongshan desk

We quote GPUs by the gigabyte now. On H100 and H200 lots, the first question buyers ask is price per card, and the first question we ask back is which memory configuration. A 141 GB H200 and an 80 GB H100 differ by 61 GB of HBM3e and nothing else, and that is where the entire premium sits. When a buyer says the H200 looks overpriced next to the H100, the honest answer is that they are buying 61 GB they may not need — which is a legitimate decision, but only if it is made deliberately.

The baseboard is the part nobody quotes. SXM5 deals that look like bargains usually exclude the HGX baseboard, the matching power shelf and the chassis. A complete 8-GPU Hopper node needs all three, they are generation-specific, and the availability of the baseboard — not the cards — is often what sets the delivery date. We have seen more projects delayed by a missing power shelf than by GPU allocation.

"Upgrade the HBM and it's an H200" is not a thing. Every quarter someone offers a "converted" H200 with the memory from a donor card. HBM stacks are not field-replaceable on these modules; anything described that way is a rebuild with no warranty and no provenance, and it will fail ECC under sustained load in exactly the way the verification checklist is designed to catch.

PCIe nodes lose the fabric, not just the bandwidth. Buyers comparing an 8×H100 PCIe build to an 8×H100 SXM build fixate on the 756 versus 989 TFLOPS gap. The larger difference is that PCIe cards bridge in pairs at 600 GB/s, so a tensor-parallel job across eight GPUs runs over PCIe root complexes instead. For single-GPU inference it does not matter; for anything that needs all-to-all, it decides the platform.

Before you send the RFQ

The difference between a quote that closes and one that circles for weeks is usually what was specified up front. Send these nine items and the reply comes back firm:

1. Model and form factor — H100 PCIe, H100 SXM5, H200 SXM, H200 NVL, B200 SXM. These are different products and are quoted separately.
2. Memory size — 80 GB, 94 GB, 141 GB, 180 GB. Never leave it implicit.
3. Condition grade — new factory sealed, pulled & tested, or broker refurbished, with the acceptance thresholds you require.
4. Baseboard and chassis in scope — if you are building nodes, say so; if you are buying modules into an existing HGX platform, say which generation.
5. Health data — GPU hours, aggregate ECC history, row remap state and sustained-load thermals per batch, before payment.
6. Serial list — per unit, in the PO.
7. Quantity and delivery — 8 cards, 8 nodes or a full rack changes the channel and the lead time.
8. Power and cooling — per-rack kW available at your site; this is what decides Hopper versus Blackwell as much as budget does.
9. Destination and Incoterm — export status by SKU and destination, plus FOB, CIF, DAP or DDP, stated before payment terms.

FAQ

Why are there no prices in this guide? Because accelerator quotes move week to week. Condition grade, memory configuration, batch size, test data and destination all change the number, and a printed price is stale before it ranks. We quote against held stock, list the exact configuration and condition grade on the offer, and print a validity window on every quotation (seven business days by default). Any input from this article — model, memory, quantity, power, destination — gets a firm quote back within 24 hours.

Is a used H100 still worth buying in 2026? Yes, for inference and fine-tuning where capacity per card is not the binding constraint. A pulled H100 SXM with recorded hours, clean ECC and a traceable serial list delivers rated performance at a discount to new, and it is the cheapest way into 80 GB of HBM3 with full NVLink. It is a poor purchase only when the workload needs more than 80 GB per card, in which case two H100s cost more power and more chassis than one H200.

What is the difference between H100 and H200? Only memory. The H200 uses the same GPU dies with 141 GB of HBM3e at 4.8 TB/s instead of 80 GB at 3.35 TB/s, at the same 700 W and the same 989 TFLOPS dense FP16. It exists because inference is bandwidth- and capacity-bound, not compute-bound.

Do B200 cards show up on the secondary market? Rarely, and mostly as part of a decommissioned rack rather than as loose cards. B200 is allocated through OEM and distributor channels; genuine second-hand supply in Q4 2026 is thin. Treat any offer of large B200 quantities at a discount as a provenance question first.

What goes into an 8-GPU node quote? Eight SXM modules, one HGX baseboard, two host CPUs, 32 DDR5 RDIMMs for 2 TB, eight 400G NICs, eight NVMe drives, and a chassis with the matching power shelf and cooling. The baseboard, power shelf and chassis are generation-specific; the memory, drives and NICs are not. Quote the node, not the cards, or the missing parts will set your delivery date.

Can I mix H100 and H200 cards in one cluster? Not usefully within a training job — tensor and pipeline parallelism assume identical memory per rank, and a mixed node falls back to the smaller card's capacity. Across separate pools it is normal practice: H200 for serving, H100 for fine-tuning, A100 for CI and staging. Document the mix per pool so the next owner of the fleet knows what is where.

What lead time should I expect for bulk GPU orders? Pulled H100 and H200 batches: one to three weeks from a held-stock position in Hong Kong, longer for specific memory configurations. New B200 and B300: 12–20 weeks through allocation, and the baseboard and power shelf can extend that. Check the validity printed on your quote before you commit an allocation, because stock moves faster than the paperwork.

Are export controls still a factor for these cards? Yes, and they move. Restrictions are set by SKU, performance class and destination, so the compliance answer belongs in the quotation, not in a follow-up email after payment. We state restricted SKUs and destinations explicitly on every quote and will not structure a shipment to route around controls.