What changed
Google Cloud announced the general availability of Dynamic Slicing on Google Kubernetes Engine (GKE), which decouples logical TPU slice formation from physical nodepool provisioning. This change allows users to dynamically stitch together TPU cubes using optical circuit switching (OCS) technology to form larger slice topologies (super-slicing) or partition TPU cubes into smaller slices for multiple workloads (sub-slicing).
What this means in practice
Dynamic slicing reduces AI workload startup times by up to 5x and recovery times by up to 4.5x. It eliminates the need to manually delete and recreate nodepools, allowing for zero-downtime topology agility. Users can now dynamically assemble custom TPU slice topologies on demand, improving resource utilization and enabling multi-tenant packing of smaller workloads within nodepools. Additionally, it allows for the use of partially healthy nodepools, improving fault tolerance and workload recovery.
Limitations and availability
Dynamic slicing is generally available on GKE. It is offered in two forms: super-slicing, which creates larger TPU slices by combining multiple TPU cubes, and sub-slicing, which partitions a single TPU cube into smaller, independent slices. To use dynamic slicing, users must pre-provision TPU capacity into nodepools of 16 VMs each, which maps to a TPU cube or sub-block. The feature is designed for large-scale AI workloads and supports a wide variety of heterogeneous AI workloads within the same environment.
