Intel has released a new set of tools called 'gpu-ai-skills' to simplify the process of running, benchmarking, and migrating Hugging Face models on its Arc and Arc Pro GPUs. The toolset includes 20 production-grade agent skills that guide any skills-capable agent through the entire workflow, from sizing to deployment. The tool is open-source under the Apache-2.0 license and can be installed with a single command. Users can quickly determine if a model fits on their GPU by using the model-can-it-fit skill, which calculates memory requirements and provides a verdict on compatibility. For example, the Qwen2.5-32B model in int4 format was found to fit on an Arc Pro B70 GPU with 11.2 GiB of headroom. This eliminates the need for trial and error, saving time and resources. The tool also supports benchmarking and profiling, helping users identify performance bottlenecks and optimize their models for Intel hardware. The skills are organized into categories, including setup, model running, planning, benchmarking, and migration from CUDA to XPU. The toolset is designed to make GPU model deployment more accessible and efficient for developers and researchers. The toolset is available for use and testing, offering a streamlined approach to working with Intel GPUs for AI tasks.
The 'gpu-ai-skills' toolset is structured to guide users through a series of steps, starting with setup and readiness, then moving to model execution, planning, benchmarking, and migration. Each skill is designed to handle a specific task, such as checking GPU compatibility, running models with the correct framework, or generating deployment plans. For instance, the xpu-model-type-detect skill helps identify a model's type before loading, ensuring the right AutoModel class is used. The torch-xpu-run skill allows users to run any Hugging Face model with PyTorch on Intel GPUs, while the vllm-xpu-run skill serves models with an OpenAI-compatible API. The sglang-xpu-run and llamacpp-xpu-run skills support additional frameworks and model formats, expanding the toolset's versatility. The planning and sizing skills are particularly useful for avoiding wasted GPU hours by pre-calculating memory requirements and recommending optimal configurations. These features make the toolset a valuable resource for developers looking to leverage Intel's GPU capabilities for AI workloads.
The toolset is the result of Intel's efforts to make GPU model deployment more accessible and efficient for developers and researchers. It is part of a broader initiative to support AI workloads on Intel hardware, offering a streamlined approach to working with GPUs for AI tasks. The skills are organized to guide users through a series of steps, from initial setup to deployment, ensuring a smooth and efficient workflow. The toolset is available for use and testing, offering a practical solution for those looking to leverage Intel's GPU capabilities for AI workloads. The release of 'gpu-ai-skills' reflects Intel's commitment to supporting the AI community with tools that simplify the deployment and optimization of models on its hardware.
Source: huggingface
