What changed
AMD has released validated, optimized inference support for five MONAI Model Zoo bundles on AMD Instinct GPUs. This is an Early Access release focused on inference, and the optimization is achieved through a ROCm overlay. The five bundles include tasks such as 3D segmentation, multi-organ CT segmentation, and spleen annotation.
What this means in practice
The validated bundles run through the standard monai.bundle runner and clear their upstream Dice thresholds without changes to model weights or inference logic. The optimizations include channels-last 3D memory layout, BF16 automatic mixed precision, and torch.compile() using PyTorch’s Inductor backend. These optimizations are applied consistently across all five bundles, with the exception of spleen_deepedit, which adds a line to its own overlay.
The performance measurements were taken on an AMD Instinct MI355X (gfx950) with ROCm 10.0, using a spleen CT volume (spleen_10.nii.gz). Per-bundle compiled-PyTorch latencies range from 3.60 ms (spleen_deepedit) to 3,287 ms (pancreas_dints).
The overlay merges into the base configuration at load time, and no upstream files are modified. Configuration files are either in JSON or YAML format, depending on the bundle.
Limitations and availability
The validated bundles are available in the AMD fork of the MONAI Model Zoo in AMD-Ecosystem/monai-model-zoo. To run them, users need an AMD Instinct GPU, Python 3.12, MONAI 1.6.0, and ROCm 10.0. The measurements in this post were taken on MI355X; MI300X and MI325X are supported by the release. The overlay applies channels-last 3D, BF16, and torch.compile() to the stock bundle configuration without requiring offline engine compilation. The optimizations are applied through a ROCm overlay, and the bundles are run with the standard monai.bundle runner.
