A recent r/SEO post highlighted Cloudflare's unexpected blocking of certain AI crawlers, causing confusion among website owners. The blocking is primarily managed through Cloudflare's "managed robots.txt" feature, which aims to prevent AI crawlers from accessing content. However, the specific directives in the robots.txt file are often misinterpreted, leading to incorrect assumptions about which AI services are affected.Cloudflare's managed robots.txt file explicitly lists eight user agents, with a note that the one deciding ChatGPT citations is not among them. While the file mentions blocking GPTBot for training data, it does not prevent AI search crawlers from accessing content. The Google-Extended agent, often mistakenly thought to control AI Overviews, is stated by Google to not impact search inclusion or ranking.The content-signal line, search=yes, ai-train=no, use=reference, is a Cloudflare convention and does not grant permission for AI-generated summaries. Google's AI Overviews are controlled by standard snippet controls like nosnippet or noindex, not specific AI crawler directives. It's crucial to check robots.txt over the network via curl or a browser, as the edge-assembled file differs from the one on disk.Cloudflare's managed robots.txt is activated by the "Block AI bots" toggle. This toggle was introduced on July 1, 2025, and new domains are now asked about AI crawler permissions during onboarding. A recent update on September 15, 2026, will introduce new defaults blocking training crawlers on pages with ads, affecting multi-purpose crawlers like Googlebot for those with the legacy "Block AI bots" option enabled.Bot Fight Mode is a separate feature that issues challenges instead of modifying the robots.txt file, and it operates independently of AI blocking settings. Perplexity was de-listed as a verified bot by Cloudflare and is blocked via heuristics, not robots.txt directives. Users should be aware of potential misinterpretations of Cloudflare's AI bot blocking features and verify their settings directly.