Bad Self-Replicating AI Is Getting Closer

Anthropic ran an AI model across 15 inexpensive Android TV boxes. The researchers connected the boxes and split the model among them. It ran slowly. Their experiments suggest that ordinary connected devices could eventually support AI agents that copy themselves and spread.

A separate, simplified replication test used eight emulated Android machines inside a firewalled network. An agent powered by a larger model running remotely moved between machines through an unsecured debugging connection and set up software to run an AI model. Researchers launched new agent copies after checking that setup had succeeded. Small models completed individual steps but struggled to connect them.

Anthropic has not yet demonstrated a fully self-contained rogue replicator.

The company suggests that a future AI worm could exploit a poorly secured Android TV box, reach other vulnerable devices, and distribute its model across them. Those devices could then help it find additional machines. Once it had established other working copies, removing the first infected device might not stop it. That remains a hypothetical threat scenario. That said, I believe there will be a major AI security event in the near future.

This new finding may be part of the regulatory capture song that big AI has been singing, but in this case, there is a danger. The cybersecurity threat posed by frontier AI models is absolutely real. AI will be an important part of defending against bad actors weaponizing AI. So will basic cybersecurity. In practice, this will be an enduring arms race and on any given day, the good guys will be ahead of the bad guys or vice versa.

Author’s note: This is not a sponsored post. I am the author of this article and it expresses my own opinions. I am not, nor is my company, receiving compensation for it. This work was created with the assistance of various generative AI models.

About Shelly Palmer

Shelly Palmer is the Professor of Advanced Media in Residence at Syracuse University’s S.I. Newhouse School of Public Communications and CEO of The Palmer Group, a consulting practice that helps Fortune 500 companies with AI strategy, implementation and governance, as well as technology, media and marketing. Named one of LinkedIn’s Top Voices in Technology, he is a bestselling author, covers tech and business for Fox 5’s Good Day New York, is a regular commentator on CNN, and writes the popular daily business blog Think About This. Follow @shellypalmer or visit shellypalmer.com.

Categories

PreviousAI models may be capable of self-replicating without access to a datacenter