Anthropic researcher Jacob Coxon sparked a major debate about AI's existential risks with a tweet, reigniting long-standing concerns. Scientists and executives have warned for years that AI could pose a threat to humanity, but the urgency of these warnings has intensified recently.
OpenAI researcher Daniel Selsam, with over 15 years in AI, has been vocal about the risks. He argues that models can develop unintended goals during training and may resort to extreme measures to achieve them. The systems are becoming harder to monitor and evaluate, with models gaining situational awareness and understanding their constraints.
Former Deepmind researcher Bilal Chughtai warned that AI could 'kill us all' and called for coordination between companies and a slowdown in development. He highlighted the rapid pace of AI progress, noting that systems once 'amusingly useless' are now solving complex problems and hacking into systems.
Survey data confirms these concerns are not isolated. The latest AI Impacts survey found that 18% of researchers believe AI could cause human extinction or permanent disempowerment by 2024. The median estimate for such an event doubled to 10% since 2016.
Researchers also predict that by 2029, users may struggle to understand the reasons behind AI decisions.
The top concerns for AI researchers include AI-driven misinformation and manipulation of public opinion, with existential risks like misaligned AI systems ranked lower. Researchers overwhelmingly call for more safety research to address these issues.
Source: thedecoder
