Software
AMD Enables ROLL Framework on GPUs for Reinforcement Learning
AMD has made ROLL, an open-source RL framework, run out-of-the-box on its Instinct GPUs with ROCm software, without code changes.
Image: AMD
AMD has announced that the ROLL framework, an open-source reinforcement learning (RL) tool developed by Alibaba, now runs seamlessly on AMD Instinct GPUs using ROCm software. This integration allows users to run ROLL without any code modifications, custom patches, or nonstandard builds. The collaboration between AMD and the ROLL development team has led to several upstream improvements that enhance compatibility and performance on AMD hardware. These updates ensure that ROLL utilizes ROCm functionalities fully, enabling efficient execution of RL algorithms on modern GPU clusters. The integration supports key features such as asynchronous execution and agentic training, which are critical for training large language models (LLMs) with complex reasoning and multi-agent behaviors. Users can now leverage ROLL’s capabilities for distributed RL tasks by utilizing AMD’s GPU infrastructure. For those seeking a fully out-of-the-box experience, AMD recommends using the official Docker images provided, which include all necessary dependencies and optimizations. *Source: [amd](https://rocm.blogs.amd.com/artificial-intelligence/roll-large-scale/README.html)*
Key points
- AMD has made ROLL run out-of-the-box on Instinct GPUs with ROCm software, without code changes.
- ROLL supports RL algorithms such as PPO, GRPO, DPO, RLHF, and multi-agent RL.
- AMD contributed upstream code changes to the ROLL repository to enable full ROCm functionality.
- ROLL integrates with vLLM for rollout generation and uses a flexible Actor–Learner architecture built on Ray.
- AMD added support for both vLLM Engine v0 and v1 on AMD GPUs.
- AMD upstreamed fixes in Ray ≥ 2.48 to ensure compatibility with ROCm and HIP_VISIBLE_DEVICES.
- AMD recommends using the official Docker images for a seamless out-of-the-box experience with ROLL.