BAAI has released Orca, a world foundation model that matches the performance of specialized robotics systems across five tasks without ever seeing a single action label. The model builds an abstract internal representation of the world from image and language signals, then uses separate modules to generate text, images, or robot movements. Orca combines two learning methods: one that watches unlabeled videos and another that learns from described actions. The approach could help address the chronic data shortage in robotics. | Image: BAAI
Orca's training data includes 125,000 hours of video footage, 160 million event descriptions, and 11.5 million question-answer pairs. The videos span four views, covering first-person shots of everyday interactions, third-person object handling, and naturally occurring scenes. Only one-tenth of the video data was used in the current version. The model's frozen core, based on Qwen3.5, remains unchanged across all output types, with separate modules handling text, images, and robot actions. | Image: BAAI
The team emphasizes that Orca is not designed to chase top scores on single benchmarks but to demonstrate how a well-trained internal world state can serve as a shared base for diverse tasks. Orca-4B, the model's largest variant, outperforms several VLM baselines on text benchmarks and image prediction tasks. It also matches a specialized robotics system, π0.5, in five manipulation tasks using a two-armed humanoid robot. | Image: BAAI
Source: thedecoder
