World Labs, the startup founded by AI pioneer Fei-Fei Li, has unveiled a simulation engine that trains robot control systems entirely in virtual environments. The models then run reliably for hours on real hardware. The company's 'Real-to-Sim-to-Real' (R2S2R) engine turns real-world robot tasks into simulations for training and evaluating control models, cutting out expensive tests on actual hardware. The technology comes from SceniX, a startup World Labs acquired in July.
One real-world task becomes thousands of controlled variations. The engine captures robots, sensors, the environment, and task demos, then rebuilds them as an interactive virtual world that doesn't just look like the original but behaves the same way physically. World Labs pulls this off by combining generative world models with task-oriented robot simulation. From a single real-world task, the system generates thousands of variations by changing lighting, object position and count, the surrounding environment, physical properties like friction, and camera angle. To check accuracy, World Labs runs the same action sequence in simulation and reality side by side and compares observations, object movements, and outcomes.
The examples shown include rigid, movable, and deformable objects such as cable routing, inserting an elastic cable end into a hole, and packing a box with both hands. Control models that never trained on real hardware train in simulation and then transfer to real robots. One of the test platforms was ALOHA, an open-source dual-arm design from Stanford operated through puppeteering with two smaller control arms. The setup costs a fraction of commercial systems, and all blueprints are public, making ALOHA the go-to reference platform in robotics research.
Source: thedecoder
