Trillium Labs, founded by Nathan Lambert and Tom Zick, launched an open-source AI research initiative focused on high-risk areas like recursive self-improvement (RSI) and reinforcement learning. The nonprofit aims to increase transparency in AI development, allowing outside experts to scrutinize and replicate experiments.
The initiative comes as the AI industry debates whether to keep powerful models locked within labs or share them openly. Lambert argues that the current closed approach to AI development is taking humanity back in time, undermining the scientific method that has historically mitigated harms.
Trillium Labs will initially focus on post-training fine-tuning and RSI, areas that have raised concerns among researchers about the potential for AI to gain unchecked autonomy. Zick emphasized the importance of publishing details on reinforcement learning, noting that outside scrutiny could lead to unexpected insights.
"Over the past few millennia, humanity has had the scientific method in our toolbox as a way to mitigate harms and build better futures," Lambert told WIRED. "The current closed trajectory of frontier AI development is taking us a step backwards."
The nonprofit has raised an undisclosed sum from Schmidt Sciences, Halcyon Futures, and others, with plans to raise $40 to $100 million in total. Lambert and Zick hope their work will add nuance to the broader conversation about AI safety and development.
Trillium Labs did not specify how it will handle potential risks associated with its research, and the open approach raises questions about the balance between innovation and safety. The founders say they plan to spend $30 million on training over the next 18 months.
Source: wired
