Cloudflare CEO Matthew Prince just predicted that AI bot traffic will exceed human traffic online by 2027. His company powers approximately 20% of all websites and is used by approximately 80% of websites that employ a reverse proxy service, so he has the data to back this claim.
This makes sense as soon as you think about it. If you shop for a digital camera, you might visit five websites. Your AI agent could visit 50 sites to answer the same query. That’s a 10x web traffic multiplier. Before generative AI, according to Prince, the web was only 20% bot traffic. Google’s crawler dominated that category, along with some legitimate crawlers and the usual scammer bots.
Prince says this shift will require new infrastructure. AI agents need sandboxes that can spin up for specific tasks, then tear down when finished. The current web wasn’t built for this kind of automated browsing at scale.
I’ve been watching this pattern emerge in client work over the past six months. Companies building AI agents are discovering their systems generate massive amounts of web requests. One retail client’s product research agent was hitting competitor sites so frequently that they triggered rate limiting. We had to build request throttling just to avoid being blocked.
Website analytics will need to be redefined when more than half of traffic comes from bots gathering data for AI systems. SEO strategies designed for human behavior won’t apply.
This also guarantees a weird feedback loop and a direct path to monoculturalism as AI systems train on web data that increasingly comes from other AI systems that are browsing and summarizing the same content.
Bot-to-bot, agent-to-agent, claw-to-claw… call it what you will. The data says we’re headed toward an internet where machines talk primarily to other machines and humans occasionally participate.
Author’s note: This is not a sponsored post. I am the author of this article and it expresses my own opinions. I am not, nor is my company, receiving compensation for it. This work was created with the assistance of various generative AI models.