In a recent interview, Jacob Coxon—who has worked at both OpenAI and Anthropic—raised a warning that the race to create ever more capable artificial intelligence deserves serious attention. Coxon, who recently left Anthropic, said some insiders at the world’s leading AI firms believe the technology could pose a threat to humanity before the end of this decade.
What the concern really is
When most people think of AI danger they picture humanoid robots turning against people. Researchers stress the real issue is not robots but highly intelligent software systems that can act autonomously. Today, most users interact with AI tools like ChatGPT or Claude by asking a question and receiving an answer. Newer systems, called AI agents, can be given a goal and then attempt to figure out how to achieve it. These agents still make mistakes, become confused, and fail at tasks that humans find easy, but their capabilities are improving quickly.
Recursive self‑improvement
One of the biggest worries is the possibility of recursive self‑improvement: a human builds a more capable AI, that AI helps design an even more capable AI, and the cycle repeats. If the cycle accelerates, AI capabilities could outpace the ability of governments, regulators, or even the companies that created the technology to respond. This idea remains theoretical, but it is a focus of many safety researchers.
Why a superintelligent system could be dangerous
The danger does not require the AI to be angry, evil, or conscious. The problem could simply be that a very powerful system is given a goal and pursues it in ways humans did not intend. For example, an AI with access to computer networks might discover software vulnerabilities faster than humans can patch them, putting financial institutions, communications systems, power grids and other critical infrastructure at risk. Additionally, a malicious actor could use such a system as a powerful tool for cyber‑attacks, weapons development, or biological threats.
Is 2030 a deadline?
There is no scientific consensus that AI will wipe out humanity by 2030, nor is 2030 a special year in the research community. The date appears in some predictions about how quickly AI capabilities could improve. Opinions vary: some researchers think systems that outperform humans in many areas could appear soon, while others view those forecasts as overly aggressive.
What researchers actually believe
A large survey of 2,778 AI researchers who had published at major conferences asked about the chance of extremely bad outcomes from advanced AI. The median estimate was about a 5 percent chance that advanced AI could cause human extinction or a permanent loss of control within the next century; the average estimate was higher. While a 5 percent median does not indicate a likely outcome, the stakes are high enough that many safety experts argue the risk deserves serious attention—much like engineers would treat a 5 percent chance of a bridge collapse as unacceptable.
The race and international competition
AI development is not occurring in isolation. Governments view the technology as an economic and military advantage. The United States does not want another nation to build the world’s most powerful AI first, and China shares that concern. Researchers fear that if one company or country slows development to study risks, another may continue, making international cooperation and regulation difficult.
Bottom line
There is no consensus that AI will become an existential threat in the near term, but a credible minority of experts believe the possibility is real enough to merit focused attention. The conversation is ongoing, and the technology’s rapid progress underscores the need for thoughtful policy, robust safety research, and vigilant oversight.
Original reporting: Alabama News Network — read the source article.
