What changed
LeRobot v0.7.0 introduces interactive rollouts, enabling users to communicate with robots in real time using text-based commands and policies. The update also expands datasets and training capabilities, including support for training directly from the Hub, Storage Buckets, or LanceDB without downloading.
New world action models and virtual language agents (VLAs) are now available, along with enhancements to the training stack for multi-billion-parameter models.
What this means in practice
Users can now interact with a running robot by altering its course, asking the model what it currently sees, or assigning it a new goal. This is achieved through text-generating policies such as FineART-VLA and Wall-OSS-0.5, or an external VLM planner.
The training stack has been rebuilt to support multi-billion-parameter world action models, with features like FSDP2, HSD, distributed checkpoints, and support for FP16, EMA, and Trackio.
Policies can now run on a remote GPU using lerobot-policy-server, and datasets stream from the Hub, Storage Buckets, or LanceDB without downloading. Simulators now run on one core, locally or served.
Limitations and availability
The update includes four new world action models (Cosmos 3, Long-WAM, FLUX 3 Action, LaWAM) and four new VLAs (G0.5, LingBot-VLA 2.0, DM0.5, Wall-OSS-0.5). The Hub now holds almost 300 TB of open Physical AI data in LeRobot format.
Interactive rollouts require a policy that generates text, such as FineART-VLA or Wall-OSS-0.5, or an external planner. The Unitree G1 robot now uses NVIDIA's SONIC whole-body controller.
Training with language annotations is supported through YAML recipes that turn lerobot-annotate labels into chat-style training targets. The update is available for use with the latest LeRobot framework.
