Rishub Jain, a former Google DeepMind researcher, resigned after growing concerns about the unchecked development of AI. He feared that the technology's self-improvement capabilities could lead to dire consequences for humanity.
Jain is among a growing number of AI researchers warning about the risks of recursive self-improvement, a process where AI systems enhance themselves indefinitely without human oversight. The idea of AI surpassing human control has caused significant alarm within the research community.
"AI progress is increasing, and as AI becomes more capable, it poses more risks," Jain told WIRED. He emphasized that the lack of visibility into how AI models evolve makes him uneasy about the potential outcomes.
The panic has intensified as AI capabilities have advanced rapidly, with recent breakthroughs raising concerns about security threats. Researchers like Jacob Coxon have resigned from companies like Anthropic, warning that the race toward self-improving superintelligence is dangerous.
"We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," said a senior Anthropic leader. This sentiment reflects the growing unease among experts about the trajectory of AI development.
Despite these concerns, not everyone sees doom as inevitable. Jain has launched Sampura Research to develop techniques that keep humans in the loop while AI handles most tasks. He remains hopeful that AI can be controlled with the right safeguards.
Source: wired
