AMD has expanded its Quark software toolkit to support SVDQuant and Hugging Face Diffusers, enabling more efficient diffusion model inference. The updates allow for 4-bit quantization of both weights and activations, which significantly reduces memory usage and latency while maintaining image quality. According to the company, these enhancements are part of its ongoing efforts to optimize AI workloads on AMD hardware.
SVDQuant, a new quantization method, addresses challenges in 4-bit activation quantization by using a combination of smoothing and low-rank correction. This approach helps preserve accuracy while reducing computational overhead. The technique is implemented through SVDQuantConfig, with multiple quantization modes available, including w4a4, mxfp4, and nvfp4. On the MI350 GPU, native inference with SVDQuant achieves a 0.92× speedup over BF16 while cutting memory usage by about 27%.
The updates also include native support for Hugging Face Diffusers, allowing quantized models to be saved and reloaded using standard Diffusers APIs. This simplifies the workflow for developers and enables offline quantization, where models can be quantized once and reused across different environments. The changes are part of AMD's broader strategy to improve AI performance on its hardware.
Source: amd
