Article URL: https://www.pacingthefrontier.com/ Comments URL: https://news.ycombinator.com/item?id=49089240 Points: 17 # Comments: 6

AI could help create a dramatically better future, but that outcome is not guaranteed. The world's leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems. To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress. "Signed because this statement helps establish common knowledge about the possible need for coordination mechanisms as automated AI research accelerates progress. I'd also like to see labs start designing these mechanisms voluntarily, even before the USG gets involved." "AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence. This will lead to unprecedented social and safety risks. To ensure a positive future we need to develop AI in a way that is driven by responsibility and thoughtfulness." "Frontier AI capabilities are advancing rapidly, and the pace of progress is itself accelerating. We see this directly in our evaluation work. CyberGym and ExploitGym show that frontier AI agents are now capable of discovering and exploiting real-world software vulnerabilities, which without appropriate safeguards, could enable cyberattacks at scale. Many researchers also consider recursive…"Show more "If it's true that AI will be as transformational as electricity, we better be intentional about how it is introduced to the world. A mindless race without assurances on governance, safety or the technology serving the public interest stands in the way of such intentionality." "I have worked in AI for a decade. Every few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions. AI may end up progressing far more quickly than societies and states can adapt. Thus, many prominent leaders in AI have publicly supported the ability to slow down, including DeepMind CEO Demis…"Show more "In the span of four years, we have gone from people having their first experience of an Al that could understand language, to Al that is performing superhuman feats of software engineering, making frontier math breakthroughs, and accelerating scientific research. Progress has shown no signs of slowing down and by default it is in the interest of individual corporations and countries to push the…"Show more