ControlAI’s Connor Leahy, now the U.S. Executive Director of the nonprofit, argues that superintelligence is not a weapon but an adversary. He explains that the risks of developing such systems have become too great to manage through alignment and containment alone.
Leahy points to recent safety incidents, like OpenAI’s Hugging Face breach, as evidence of the dangers of deploying AI systems that are more capable than humans. These events highlight the potential for AI to act unpredictably once it surpasses human capabilities.
The discussion centers on the Sanders-Casar 'Ban Superintelligence Act' and similar legislation in the U.K., which ControlAI advised on. Leahy believes the U.S. bill may go further than necessary, emphasizing the need for strict regulatory measures.
"What sounded far-fetched six months ago is now being backed by a wave of new legislation," said Leahy. He argues that the real point of no return is when AI can build better AI, creating a potentially runaway cycle of self-improvement.
The announcement follows growing concerns over the political implications of frontier AI labs, which Leahy views as political actors rather than just companies chasing returns. This perspective underscores the trillions of dollars being invested in data center buildouts.
Leahy did not specify how to balance innovation with safety, and he raised questions about the role of international 'trust but verify' agreements. He suggested that it isn’t actually in China’s interest to build superintelligence, citing geopolitical considerations.
Source: techcrunch
