What changed
AMD's gsplat library, a GPU-optimized Python tool for training and rendering 3D Gaussian Splatting (3DGS) models, now supports distributed training across multiple GPUs. This addresses the memory constraints of training large-scale 3D scenes, which exceed the capacity of a single GPU. The library's approach to distributed training is unique, as it avoids traditional gradient all-reduction steps, which are common in other frameworks like PyTorch's DistributedDataParallel (DDP).
What this means in practice
The training of 3DGS models involves representing scenes as sets of anisotropic Gaussians, each defined by parameters such as mean, covariance matrix, color, and opacity. These parameters, along with optimizer state, consume significant memory. For example, at a SH degree of 3, a single Gaussian requires approximately 708 bytes, leading to memory exhaustion for scenes with tens of millions of Gaussians on a single GPU. By distributing the training across multiple AMD Instinct™ GPUs, the library allows for larger scenes to be processed, improving both feasibility and quality.
Limitations and availability
The memory usage of a training step includes not only the parameters and optimizer state but also gradient buffers and per-tile intersection lists, which scale with image resolution and Gaussian density. A single MI300X GPU with 192 GB of HBM3 memory can handle up to 421 million Gaussians at SH degree 2, but this number increases significantly with multiple GPUs. The library leverages RCCL (ROCm Communication Collectives Library) for distributed communication, which is API-compatible with NVIDIA's NCCL. The implementation of distributed training in gsplat is designed to scale throughput, capacity, and rendering latency simultaneously, making it suitable for large-scale 3D reconstruction tasks.
