NVIDIA GB300 NVL72 — Liquid-Cooled Data Center Rack

48U MGX rack  ·  18 compute trays  ·  9 NVLink switch trays  ·  CDU cooling loop  ·  72× B300 GPUs  ·  135 kW TDP  ·  1.1 ExaFLOPS FP4

Raised floor · Reinforced concrete · 21 kN/m² rated · ~3,000 lb rack weight NVIDIA GB300 NVL72 · 48U MGX Rack 1U 5U 9U 13U 17U 21U 25U 29U 33U 37U 41U 45U 48U Coolant Distribution Manifold Header DI water supply / return · QD fittings · leak detection sensors NVLink Switch Tray 1 — 5th-gen NVLink · 1.8 TB/s per GPU B300 B300 B300 B300 cold plate CX8 CX8 CX8 CX8 BF3 Grace×2 Tray 1 B300 B300 B300 B300 CX8 CX8 CX8 CX8 BF3 Grace×2 Tray 2 NVLink Switch Tray 2 B300 B300 B300 B300 CX8 CX8 CX8 CX8 BF3 Grace×2Tray 3 B300 B300 B300 B300 Grace×2Tray 4 NVLink Switch Tray 3 B300 B300 B300 B300 Tray 5 B300 B300 B300 B300 Tray 6 NVLink Switch Tray 4 B300 B300 B300 B300 Tray 7 B300 B300 B300 B300 Tray 8 NVLink Switch Tray 5 B300 B300 B300 B300 Tray 9 B300 B300 B300 B300 Tray 10 NVLink Switch Tray 6 B300 B300 B300 B300 Tray 11 B300 B300 B300 B300 Tray 12 NVLink Switch Tray 7 B300 B300 B300 B300 Tray 13 B300 B300 B300 B300 Tray 14 NVLink Switch Tray 8 B300 B300 B300 B300 Tray 15 B300 B300 B300 B300 Tray 16 NVLink Switch Tray 9 (final) B300 B300 B300 B300 Tray 17 B300 B300 B300 B300 Tray 18 Power Shelves × 8 (33 kW each = 264 kW capacity) PSU 1 PSU 2 PSU 3 PSU 4 PSU 5 PSU 6 PSU 7 PSU 8 DC Bus Bar — 48V DC distribution to all trays IEC 60309 5-wire whips · 135 kW TDP · 155 kW peak · provision to 192 kW In-Rack CDU — Coolant Distribution Unit Liquid-to-liquid HX · 250 kW capacity Facility water IN 18–22°C / OUT 40–45°C · 50 µm filtration SUPPLY RETURN Cable management · 800G OSFP LPO fiber → Arista 7060XE7 LPO optics: no DSP retiming · ~60% lower power · MRC plane wiring COLD SUPPLY ← → HOT RETURN Arista 7060XE7 MRC Fabric Switches 128×800G · 102.4 Tbps · 267MB buffer Leaf P1 — 9× 800G server downlinks Leaf P2 — 9× 800G server downlinks Leaf P3 — 9× 800G server downlinks Leaf P4 — 9× 800G server downlinks Leaf P5 — 9× 800G server downlinks Leaf P6 — 9× 800G server downlinks Leaf P7 — 9× 800G server downlinks Leaf P8 — 9× 800G server downlinks ↑ uplinks to 8× spine switches No cross-plane links — fully independent MRC 32-bit EV / SRv6 uN SID · 128–256 EVs per QP · zero ECMP · no dynamic routing LPO optics · 840 ns latency 7060XE7 64×1600G / 128×800G 800G OSFP LPO Power & Thermal 135 kW TDP / 155 kW peak 1,400 W per B300 GPU ~3,000 lb / ~1,360 kg total weight GPU Memory 288 GB HBM3e per B300 21 TB total HBM per rack 8 TB/s memory bandwidth / GPU Compute 15 PFLOPS FP4 per B300 1.1 ExaFLOPS FP4 / rack 130 TB/s NVLink / rack MRC Networking 8× CX-8 SuperNIC @ 800G per tray 57.6 Tbps aggregate fabric 2-hop any-to-any · zero ECMP · no dynamic routing In-Row CDU 1.8 MW capacity for 8-rack SuperPOD cluster 576 GPUs · 11.5 ExaFLOPS Facility chilled water loop Hot / cold aisle containment Row busway: Starline / Legrand
Compute trays
B300 GPU chip (288 GB HBM3e · 15 PFLOPS FP4 · 1,400 W)
NVLink switch tray (5th-gen · 1.8 TB/s per GPU)
Cold plate (GPU · CPU · NIC liquid-cooled)
Grace ARM CPU (×2 per tray)
Networking
CX-8 SuperNIC (800 Gb/s per port · 1:1 GPU:NIC)
BlueField-3 DPU (400 Gb/s)
Arista 7060XE7 leaf switch per plane
Cooling — hybrid DLC
Cold supply line (facility water 18–22°C)
Hot return line (exits at 40–45°C)
CDU — in-rack (250 kW) or in-row (1.8 MW)
GPUs, CPUs, NVSwitch & CX-8 NICs: liquid cold plates. OSFP modules, storage, PDB: air-cooled (hybrid architecture).
Power
PSU shelf (33 kW × 8 = 264 kW headroom)
DC bus bar (48V to all trays)
Floor busway required (Starline / Legrand) — standard PDU-per-rack cannot handle 135 kW density.
MRC wiring rule
CX-8 NIC n of every tray → leaf of plane n. Each leaf has 9× 800G downlinks (one per server) and uplinks to its own dedicated 7060XE7 spine. No cross-plane links. MRC 32-bit EV striped across UDP src port + IPv6 flow label, encoded in SRv6 uN SID. 128–256 EVs generated per QP at startup, equally split across planes. Fabric switches are stateless forwarders — zero ECMP, no dynamic routing.
72
B300 GPUs
21 TB
HBM3e total
1.1 EF
FP4 / rack
135 kW
TDP / rack
18
compute trays
9
NVLink trays
57.6
Tbps fabric
130 TB/s
NVLink BW