What changed
AWS has introduced new methods for training Qwen3 models on its Trainium hardware using the torchtitan framework. These methods include using Transformers v5 as a modeling backend, building native Qwen3 models with torchtitan, and integrating Neuron Kernel Interface (NKI) mixture-of-experts (MoE) kernels.
What this means in practice
The new configurations allow users to train Qwen3 models on a single trn2.48xlarge instance using either the AWS Deep Learning AMI for Neuron or a Neuron PyTorch training container. The training process leverages the Neuron driver, runtime, and compiler to support Trainium hardware. Users can pre-train from random initialization or fine-tune published weights, and customize model implementations and kernels for their specific workloads.
Limitations and availability
The configurations require specific versions of torchtitan and its dependencies, including torch, torch_neuronx, and transformers v5. The training examples use the versions listed in the provided table, and users are advised to pin these versions to reproduce the results. The torchtitan-neuron package must be installed with --no-deps to preserve the Neuron build of torch. The Qwen3 model artifacts must be downloaded from Hugging Face Hub and stored in a local directory specified in the training configuration. The supplied announcement does not specify the exact performance benchmarks or independent testing results for these configurations.
