AMD has introduced VSA, a sparse attention mechanism designed to accelerate video diffusion inference on its GPUs. The technology, implemented through AMD’s CK Tile library, focuses on reducing computational costs by leveraging sparse attention patterns. This approach targets the attention bottleneck in video generation tasks, where standard attention mechanisms are computationally expensive due to their quadratic complexity. The new method is part of AMD’s ongoing efforts to optimize AI workloads, particularly in video generation, which remains a significant challenge for large-scale transformer models.
VSA operates through a two-stage coarse-to-fine attention approach, which reduces computation by focusing token-level attention on selected spatial-temporal regions. The method reorders video tokens using a 3D space-filling curve to ensure spatial locality, enabling efficient block-sparse attention patterns. Additionally, the system dynamically selects which blocks to attend to based on the attention patterns, adapting to the specific characteristics of video data. This allows for significant reductions in both memory traffic and attention computation while maintaining alignment with block-sparse GPU kernels.
The blog post highlights the performance of VSA on AMD Instinct™ MI308X GPUs, where it achieved a 3.31× speedup over FlashAttention at 70% sparsity. The results emphasize the importance of hardware-efficient sparse attention mechanisms in reducing the computational overhead of video diffusion models, particularly for large-scale applications. The implementation also includes a qualitative visual check to validate the effectiveness of the approach.
Source: amd
