Remote Direct Memory Access

<p>Remote Direct Memory Access (RDMA) enables direct memory-to-memory data transfer between nodes in a cluster without CPU involvement on either end. The data path bypasses the operating system kernel, achieving latency in the single-digit microsecond range and throughput close to the wire rate of the network.</p><p>RDMA is fundamental to the performance of large-scale AI training, where gradients, activations and parameters must be synchronised across hundreds or thousands of GPUs at very high speed. MPI (Message Passing Interface) and NCCL (NVIDIA Collective Communications Library) use RDMA for collective operations such as AllReduce. RDMA is supported natively over InfiniBand and over Ethernet via RoCE. Without RDMA, multi-node AI training at scale would be severely network-bottlenecked.</p>