Cloudflare and GoDaddy Partner on AI Crawler Controls

Cloudflare and GoDaddy have partnered to provide websites with control over AI crawlers and agent identity standards. This collaboration aims to help websites defend against AI-powered bot swarms from Big Tech companies.

NET (Cloudflare) and GoDaddy have announced a strategic partnership to give website owners control over AI-powered crawlers that harvest web content for training large language models. Under the deal, GoDaddy will integrate Cloudflare's AI Crawl Control directly into its hosting platform, giving the roughly 80 million domains managed by GoDaddy a simple dashboard to decide whether bots from OpenAI, Anthropic, Google, and others can access their pages. Site owners will be able to grant access, block crawlers entirely, or enable a new "pay-per-crawl" monetization model that lets them charge AI companies for content access.

The technical backbone of the partnership is Cloudflare's Web Bot Auth system, which uses cryptographic verification to distinguish legitimate AI agents from impersonators. GoDaddy contributes its Agent Naming System (ANS), a DNS-and-PKI-based open standard for consistent naming and verification of AI agents across platforms. Together, the tools are framed as infrastructure for an "open agentic web" where content creators retain visibility and economic participation rights as AI agents proliferate.

For investors, the deal matters most as a revenue signal for NET: monetizing AI crawler traffic represents a new revenue stream that is still nascent but growing rapidly as LLM training and agentic browsing scale. GoDaddy's customer base gives Cloudflare broad distribution for AI Crawl Control without direct sales costs, while cementing Cloudflare's positioning as essential AI-era internet infrastructure — a narrative that supports premium multiples on the stock.