More than 20 leading AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, warn that automated AI research poses extreme risks. They argue that AI systems already write most of the code at the companies building them and could automate the entire AI R&D pipeline within years.
According to the study, progress that normally takes years could then happen in months. The authors warn society might not keep up, control over superhuman AI could slip away, and power balances between nations, companies, and governments could erode.
The paper joins a growing list of warnings. 42 leading mathematicians recently called for more attention to existential AI risks, several AI lab employees have voiced fears that AI could destroy humanity, and Pachocki himself said no lab has solved alignment well enough 'to continue responsibly scaling at maximum speed for much longer.'
"Although there remains much uncertainty, AI R&D automation might soon trigger one," they write. The authors urge policymakers to gain far more visibility into how AI research is being automated.
The announcement follows a wave of recent warnings about AI risks. The researchers stress that the potential for an intelligence explosion is a serious concern that demands immediate attention.
The researchers did not specify how to prevent such an outcome, and they raised questions about the pace of AI development. They emphasize the need for greater oversight and transparency in AI research.
Source: thedecoder
