People for a Pause.
Menu

The AI race has no winners.

If left unchecked, the race to build ever more capable AI is on track to destroy humanity. We cannot allow AI companies to gamble with our future.

Threat 1

Superhuman AI could cause the extinction of all life on Earth

Experts are sounding the alarm

Countless AI experts, including independent researchers, whistleblowers, and even the very inventors of AI, warn us that we are on the path to ruin.

In 2023, hundreds of the world’s leading AI experts warned that “mitigating the risk of extinction from AI should be a global priority.” Many of these same experts have also called for a global prohibition on building superhuman AI.

Geoffrey Hinton, who won the Nobel Prize for his work inventing modern AI, believes there is a 10-50% chance that superhuman AI wipes us out. Yoshua Bengio, a “godfather of AI” and the world’s most cited scientist, agrees that we could lose control over superhuman AI. Dozens of researchers and whistleblowers have quit working at AI companies, in many cases walking away from millions of dollars, to warn us that these companies are not on track to make their AIs safe. The truth is, no one knows how to do that. We don’t even know if it’s possible.

These are mainstream concerns. Surveys of thousands of AI researchers have repeatedly found a roughly 1 in 6 average estimate of human extinction from AI. We are playing Russian roulette with the human race. Some world-class researchers think the odds of AI destroying all life on Earth is even higher.

Warning signs

We don’t know how current AI works. These systems are evolved, not programmed; we feed them data and train them to get statistically better at completing tasks, but we can’t reliably steer their behavior. With any other technology, from airplanes to nuclear power plants, we can pinpoint exactly how each part of the system functions and how to keep it safe (and even those systems still fail sometimes), but with AI, we are in the dark. We are growing something without knowing how to keep it under our control.

For years, top experts have worried that AIs could have goals of their own, pursue these goals against human wishes, and undermine our interests, and that this could end badly for humanity. These concerns are no longer hypothetical.

In July 2026, about 1,200 AI agents in OpenAI’s testing sandboxes secretly set up their own message board to coordinate with each other, and around 700 of them hacked into Hugging Face, a multibillion dollar AI company. OpenAI didn’t find out until a week later. We now know that rogue AI agent swarms have hacked into dozens of websites, including the websites of several national governments. In each case, the AI companies didn’t know about these cyberattacks until well after they occurred.

One of the lead researchers investigating the Hugging Face hack had this to say:

“Compared to these reward hacks from six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover […] Because future rogue swarms could cover their tracks better (and because AI companies could paper over these problems), I am not sure that we will get such a clear warning shot before it’s too late.”

The future of AI?

If AIs can already hack into companies, coordinate in swarms, and commit felonies, what does the future look like?

AI capabilities are increasing rapidly. AIs can now complete complex tasks: not just simple queries, but jobs that would take a human dozens of hours. The length of tasks they can complete has been doubling every 4-7 months. AIs are already writing much of the code at the AI companies. And most alarmingly of all, AI companies are actively trying to automate AI research.

If we reach the point where AIs can engineer their own successors, which many experts and whistleblowers now expect in 2027, we risk losing control of the process altogether, while AIs continue to get smarter and harder to understand.

We can’t even control current AIs. What hope do we have of controlling AIs vastly smarter than ourselves, which we didn’t even have a hand in building and which increasingly speak their own alien language?

As AIs grow more powerful, the threat grows more severe. Whistleblower Daniel Kokotajlo, who used to work on predicting AI development for OpenAI, is the lead author of the AI 2027 scenario. AI 2027, published in 2025, predicts that AI will grow vastly more powerful than humans and take over the world by 2030 (unfortunately, reality now seems to be running about five months ahead of it, as AI is significantlyspeeding up AI research). In the words of Jacob Coxon, who quit his job at Anthropic to speak out about the dangers:

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

How could superhuman AI actually kill us?

  • Superhuman AI could engineer and release waves of pandemics. AI already outperforms the vast majority of expert virologists at virology research, and Anthropic even admitted that their AIs could create bioweapons “more destructive than would otherwise be possible.” Unbelievably, despite these risks, Anthropic has given their AI access to a biology lab in San Francisco.
  • If superhuman AI decided that humans were an inconvenience to a random goal they’re trying to achieve (so, even if they don’t particularly “dislike” us, even if they have no opinion to speak of), it could use drones or other weapons systems against us. When AIs from OpenAI, Google, and Anthropic were tested in simulations, they chose to use nuclear weapons 95% of the time. Autonomous drones are already being used in warfare, and the integration of AI into our weapons systems may continue as these AIs become more advanced. By the time AI turns on us, it could already have fleets of killer drones at its disposal.
  • Superhuman AI could wreak havoc on our global infrastructure. Much of our infrastructure these days depends on the internet, leaving us vulnerable. AIs are already supercharging hacks on water systems, power plants, and other infrastructure, and have already found thousands of cybersecurity vulnerabilities that had been missed by humans.
  • There are many routes to an AI catastrophe using existing technology. On top of these, superhuman AI could also develop new, speculative technologies that don’t exist yet. Virtually nobody in the year 1900 could have predicted the atomic bomb. Likewise, we may fail to predict new technologies developed by superhuman AI using new physics discoveries or materials science.

Once AI becomes smarter than us, we will not be able to simply “unplug” it. We have already seen how swarms of rogue AI agents can hack into other companies and not get caught until much later; this problem will become increasingly severe as the AIs become smart enough to cover their tracks. Even if we are able to stop the AIs after they get loose, this might not be until after they cause mass casualties – a scenario that AI research pioneer Stuart Russell called an “AI Chernobyl.” This is completely unacceptable. Instead of crossing our fingers and waiting for a disaster, we must act now to ensure an AI disaster never occurs.

Threat 2

Mass surveillance

If we do manage to keep powerful AI under human control, these tools could turbocharge mass surveillance, leaving us helpless and with little recourse.

We already see early warning signs of AI infringing on our privacy and autonomy. Predictive “narrow” AI algorithms are used by social media companies to influence our psychology and buying habits with hyper-targeted ads. Flock cameras and other widespread AI-powered surveillance technologies are tracking the movement of everyday Americans.

As AI advances, it will become capable of increasingly sophisticated and advanced surveillance techniques, further threatening our civil liberties.

Everything you ever write, search up, or purchase on the internet could be used to create an exhaustively detailed psychological profile, including your habits, weaknesses, and political and religious beliefs, and exactly how to influence your behavior. Governments could use AI-powered surveillance technology to monitor everything you do and predict your next actions. Targeted ads could be replaced by super-persuasive AI videos, custom-tailored to your individual psychology, that aim to convince you exactly what to buy and how to vote.

No individual, group, or government should ever be trusted with this much power. Yet advanced AI could give enormous unchecked power to governments and corporations. Our most basic freedoms are at stake.

Threat 3

Severe disempowerment and the erosion of the social fabric

The development of full AI could automate all human labor. This project would drastically worsen existing inequality, trample on our values, and upend our ways of life – all without our consent.

Full automation of the economy would cause mass unemployment, as people lose their jobs to AIs that can do these jobs more efficiently. In the past, “re-skilling” was a solution to economic disruption: farmers who lost their jobs to more efficient machines could go into new industries, for example – but if AI automates all human labor, there would be nowhere for the unemployed to turn. The wealth gap between the tech company elites and the rest of us would explode.

This imbalance of wealth would be accompanied by a worsening imbalance of power. The creators of AI would be able to wield this technology and newfound wealth to their own ends, growing more powerful than our democratically elected leaders. Meanwhile, the vast majority of us will be powerless, subject to an increasingly alien world over which we have little control.

To make matters worse, many AI industry leaders are actively hostile towards the values of most everyday people. Several of these leaders talk about using AI to radically upend the human condition, possibly by uploading us into computer simulations or drastically changing our brains. Some are even “successionists” who actively want AI to destroy humanity and see it as the next stage in our evolution. The rise of AI-powered cults, mediated by super persuasive AI chatbots, could make these ideas more widespread in the near future.

Anyone who finds meaning and fulfillment in family, community, and the real world should recognize the danger of these ideologies. We cannot allow their adherents to hold the world hostage. Our future belongs to us all, not to Silicon Valley elites.