Cloudflare pilot added hundreds of millions of pages to OpenAI's search index, with no citation data yet

Cloudflare and OpenAI say a research pilot helped OpenAI discover and index hundreds of millions of new web pages for ChatGPT search. The pilot ran on a participating cohort of about 1% of Cloudflare's total network. The companies shared early results on Oct. 15 in a post on Cloudflare's blog coauthored by two OpenAI staff, as reported by Search Engine Journal.

What the pilot shows, and what it leaves out

The index got bigger. That is the only result reported.

• The post does not report how often the new pages were cited in ChatGPT or how many visits they sent.

• It gives no figures for crawl volume or server load.

• Neither announcement says who participated, or whether the new pages sit on the participants' own sites.

• OpenAI evaluates and ranks indexed pages on its own, so being found through the pilot guarantees no citation.

Being in the index makes a page eligible to be cited. Eligibility is the floor of AI visibility. Whether a page gets named still depends on how OpenAI ranks it against everything else in the pool.

Which pages gain

The signals are built to find pages that "can be valuable even if few other sites link to them." The examples are plain: a washing machine user guide, current passport requirements and fees, a restaurant's menu and hours. Signals named in the July 8 announcement include how recently a page was updated, the quality of traffic to a site, and what actually changed on a page.

That points at the unglamorous pages most businesses neglect: service pages, opening hours, pricing, product documentation. These are the pages this approach is designed to surface, so keeping them current and accurate is worth the effort.

What to check now

Signals do not override your crawler settings. The post states that "a signal about a page doesn't grant permission to access it."

• If you block OAI-SearchBot, your site will not be included in ChatGPT search results, though it may still appear as navigational links.

• Allowing OAI-SearchBot while blocking GPTBot keeps you in ChatGPT search and signals that your content should not be used for training. OpenAI gives this configuration as an example on its crawler page.

• Cloudflare's AI Crawl Control, available on every plan, shows which AI services access your site and lets you allow or block each crawler.

• Your server logs also show OpenAI's crawler activity.

Cloudflare said in a July 1 post that it intends to offer broad access to the capability during the remainder of 2026. The Oct. 15 post gives no date. The companies say the next stage includes measuring the results when the AI finds new content. Until those numbers arrive, the pilot is a story about indexing, and the citation question stays open.