AMD has released a guide to enable on-premises execution of Claude Code using AMD Instinct GPUs. This approach allows developers to run large models like GLM 5.2 without sending data to external APIs. The guide demonstrates how to set up a dedicated GPU server with AMD Instinct MI355x GPUs to serve GLM 5.2 at full quality, ensuring data remains on-premises and avoiding per-token cloud costs.
The guide outlines a setup where Claude Code runs on the developer’s machine, while the model operates on a GPU server connected via an SSH tunnel. This separation allows the developer machine to remain responsive, as the heavy inference workload is handled by the dedicated server. The setup includes SGLang for model serving and LiteLLM as a translation layer, enabling Claude Code to interact with the GPU server as if it were accessing Anthropic’s API. This configuration is ideal for organizations handling sensitive data or requiring high-quality model performance.
The guide highlights the limitations of running large models on consumer-grade hardware, such as the need for 756 GB of HBM memory to load GLM 5.2, which is not feasible on typical developer workstations. By using an AMD Instinct GPU server with 288 GB of HBM3 memory and 8 TB/s bandwidth, the setup supports FP8 quantization and 8-way tensor parallelism, enabling efficient and high-quality model inference.
Source: amd
