Two weeks ago, OpenAI announced a relaunch of its robotics program, signaling a renewed focus on teaching machines to interact with the physical world. However, building capable robots requires training data that is not yet available in the same abundance as that used for language models. This gap is giving rise to new infrastructure businesses focused on data collection and processing for robotics. Unlike language models trained on vast text corpora, robots need data that captures physical interaction, which is scarce and difficult to obtain. YouTube videos and gig worker footage are low-fidelity and hard to reconcile with the physical world. XDOF is betting that the next major bottleneck in AI is not models or chips, but the data feedback loop needed to train robots. The startup aims to build data pipelines, collection tools, and annotation systems for AI labs and robotics companies, and has raised $70 million from several major venture firms to do so.

XDOF, founded by Philipp Wu and Fred Shentu, is already working with 20 customers, including several top AI labs, though it cannot name them. Wu, who previously worked on a project called GELLO, a low-cost teleoperation system, highlighted the challenges of collecting data before training models for robotics. He noted the chicken-and-egg problem of needing data to train models, which requires data collection first. The company is also partnering with UC Berkeley’s AI Research lab to release what it claims is the largest collection of high-quality robot training data ever assembled, dubbed ABC. This dataset includes 130,000 trajectories of robot manipulation data, 300 hours of simulation, and 100 hours of evaluations. The team has already used the data to train robots on benchmark tasks like folding T-shirts and loading AirPods into their cases.

The company plans to work across three tiers of a data pyramid, starting with teleoperation data collected on actual robots, followed by data from teleoperated robots, and finally egocentric data from human tasks. Wu emphasized the importance of hardware design in data quality, noting that poor camera choices can lead to subpar data affecting hand-tracking algorithms. XDOF plans to hire and train global teams of teleoperators and egocentric data operators, a labor-intensive model that raises questions about why major labs aren’t doing this themselves. Wu explained that maintaining hundreds of robots and training operators requires significant capital and operational scale, which most labs prefer to outsource. Source: techcrunch