Nearly three quarters (up to 75%) of AI training runtime can be spent on slot online , according to studies from Google and Microsoft. The network directly impacts compute utilisation.
AMD’s new network interface card (NIC) – the Pensando Vulcano 800 AI NIC – is designed to tackle this by handling high-throughput AI training and distributed inference workloads.
It slots into a standard rack and combines three 800Gbps NICs per GPU, delivering up to 2.4Tbps of total bandwidth to each GPU.
Real-world cluster performance takes more than huge bandwidth. To maximise efficiency, the Pensando combines scale-out networking between nodes with scale-across networking between GPUs.
AMD Networking describes ‘scale-out’ as “the ability to distribute a workload across multiple nodes, expanding the effective GPU pool beyond what a single server can provide”. For training, that means syncing workloads across hundreds of servers without slowdowns and for inference, it means handling heavy request traffic smoothly.
Meanwhile, ‘scale-across’ means to extend beyond the rack, where clusters span multiple data centres.