An Open Architecture for Multi-Rack Scale-Up Domains of 5000+ GPUs
Published: October 21, 2026
As models pass trillion parameters and adopt mixture-of-experts and disaggregated prefill/decode, the binding constraint is no longer FLOPS, but how many accelerators talk at memory-semantic speed in a single hop. Most scale-up domains stop at 8 to 72 GPUs, forcing parallelism onto a slower scale-out network where every hop adds latency and tail variance.
This session, presented by Eridu CTO Promode Nedungadi and Delos Data CTO Dan Daly at OCP Global Summit 2026, presents a flat, one-hop scale-up domain connecting 5k GPUs as a single fabric with every GPU reaching every other in one hop and shows why that shrinks collective-communication time, tightens token-generation tail latency, and lets a frontier model fit entirely inside a single domain.
We detail a practical implementation: a GPU server bringing all scale-up I/O to the front panel, making a domain this size deployable through cabling and high radix switch, not a fixed backplane. We close on how open, OCP-aligned optics and standards can deliver this performance at scale. Watch the complete presentation.
