Hugging Face has released VLANeXt, an open-source codebase designed to support robotics research by offering a range of baselines for building strong vision-language-action (VLA) models. The project aims to address the fragmentation in the field by providing a systematic approach to VLA development. VLANeXt builds on a simple RT-2-style baseline and expands it to include a variety of emerging directions in robotics foundation models. This includes smaller and larger VLA backbones, latent action learning, and world-action modeling. The codebase now offers a family of representative baselines, such as VLANeXt-LAM, VLANeXt-S, and VLANeXt-WAM, which serve as clean starting points for exploring new ideas across different model scales and learning objectives. Source: huggingface
The project began with a simple RT-2-like baseline that used a third-person-view image and a language instruction to predict the robot's next action by reusing text tokens as action tokens. This initial approach was limited in performance, prompting the team to run over 500 experiments to explore various VLA design choices. These experiments focused on foundational components, perception essentials, and action modeling perspectives, leading to the development of several practical insights that guided the creation of VLANeXt. The team tested multiple strategies, including using a dedicated policy module, predicting action chunks, and modeling actions continuously. They also evaluated the impact of stronger VLM backbones and the use of proprioceptive inputs. Source: huggingface
The source text describes the development of VLANeXt as a systematic effort to improve VLA models by addressing key design choices. It highlights the team's exploration of different policy modules, action prediction methods, and the integration of proprioceptive inputs. The project's goal is to provide a flexible and extensible codebase that supports ongoing research in robotics foundation models. Source: huggingface
