Cloudflare CEO Matthew Prince has warned that bot traffic has overtaken human traffic on the internet. According to data from Cloudflare Radar, bots now account for 57.4% of HTTP requests worldwide, compared to 42.6% from humans. Prince noted that this shift happened over the past few months, with the data still being rough but the trend unmistakable. He did not expect this tipping point until late 2027, but rapid growth in automated traffic from AI agents moved the timeline up significantly. Prince emphasized that the label of bot, crawler, or agent depends on whether one views them as good or bad, but all refer to the same type of traffic. He added that the future of the web is clearly moving toward a 'pay to crawl' model, as automated traffic continues to dominate. Prince highlighted that Cloudflare launched a platform last summer to let site owners gate AI crawlers and charge for access, but it has not yet gained traction. His company is still working on 'protocols and infrastructure to support the volume it'll require.'
Prince's comments come as Google's AI Overviews and AI Mode have attracted billions of users, signaling a broader shift toward AI-driven web interactions. The rise of bot traffic has prompted companies to rethink how they manage and monetize online access. While the exact impact of this shift remains unclear, Prince's warning underscores the growing influence of automated systems in shaping the web's future. The transition to a 'pay to crawl' model could have significant implications for website owners, content creators, and users alike.
Prince stated, 'Bot, crawler, and agent all mean the same thing; the label just depends on whether you see them as good or bad.' He also noted, 'Clearly it's going to be pay to crawl.' These remarks reflect a broader industry concern about the increasing prevalence of AI-driven traffic and its potential to reshape online engagement. The data from Cloudflare Radar suggests that this trend is already well underway, with no signs of slowing down.
Source: thedecoder
