In short
- GPTBot crawls content that may be used to train OpenAI's generative AI foundation models. ChatGPT search uses third-party search providers and content provided directly by partners, while OAI-SearchBot surfaces websites in search features.
- In a July 2026 study of 1,249 answers, RESONEO found 63.6% of free instant mode results came from OpenAI's
"labrador"index, while paid thinking mode drew 74.6% from the scraping provider"bright". - Disallowing GPTBot tells OpenAI the content should not be used for training; blocking GPTBot does not remove a site from ChatGPT search; sites opted out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links; after a robots.txt change, search can take about 24 hours to adjust.
With 1.2 billion weekly users (OpenAI, September 29, 2026) and 24% of messages "Seeking Information" (September 15, 2025), checking the sources behind an answer can help you investigate outdated business facts.
How ChatGPT's two information systems work
ChatGPT combines a pre-trained foundation model with a real-time retrieval system that searches the web for current facts. When ChatGPT search launched on October 31, 2024 (opening to everyone without sign-up on February 5, 2025), OpenAI explained that it pairs a fine-tuned model with third-party search providers and direct partner feeds.
According to OpenAI's help center in October 2026, ChatGPT may search the web automatically when a question would benefit from current information, across Free, Go, Plus, Pro, Business, Enterprise, Edu and logged-out sessions.
| Feature | Foundation model training | Live web search | On-demand user fetching |
|---|---|---|---|
| OpenAI bot | GPTBot | OAI-SearchBot | ChatGPT-User |
| User agent token | GPTBot/1.4 | OAI-SearchBot/1.4 | ChatGPT-User/1.0 |
| Primary role | Crawls content that may be used to train generative AI foundation models | Surfaces websites in ChatGPT search features | Fetches a page when a user or Custom GPT asks |
| Published IP ranges | openai.com/gptbot.json | openai.com/searchbot.json | openai.com/chatgpt-user.json |
| Robots.txt effect | Opts content out of model training | Opted-out sites are not shown in search answers, though navigational links can remain; search can take about 24 hours to adjust after a robots.txt change | Rules may not apply because a user initiated it |
Understanding how ChatGPT search works starts with separating these three bots. As OpenAI documented in October 2026, the bot settings are independent, though if a site allows both OAI-SearchBot and GPTBot, OpenAI may use a single crawl for both.
Source 1: Training data and GPTBot
OpenAI uses GPTBot (openai.com/gptbot.json) to crawl web content that may be used in training its generative AI foundation models, while Common Crawl's CCBot builds an open crawl corpus widely used for LLM training data.
Disallowing GPTBot in robots.txt tells OpenAI the content should not be used for training. GPTBot and OAI-SearchBot settings are independent.
Source 2: OpenAI's search index and third-party providers
When ChatGPT search partners with other search providers, it typically rewrites the query into one or more targeted queries sent to those providers. In an August 20, 2025 study of 118,931 fan-out queries, Ahrefs found ChatGPT ran 1.78 searches per prompt on average, and 83.39% of retrieved results did not appear in Google's results at all (6.82% were in Google's top 10).
Sources of ChatGPT search information include:
1. OpenAI's own index (OAI-SearchBot and "labrador")
OpenAI crawls the web for search with OAI-SearchBot (openai.com/searchbot.json). In a July 2026 discovery pilot, Cloudflare reported that OpenAI indexed hundreds of millions of new pages through about 1% of Cloudflare's network by October 15, 2026. Between May 21 and July 21, 2026, Peec AI observed ChatGPT labeling results from OpenAI's own index as "labrador".
2. Bing and Microsoft Web IQ
At the October 31, 2024 search launch, OpenAI's Srinivas Narayanan said: "We use a set of services and Bing is an important one." OpenAI's Enterprise and Edu docs also note that ChatGPT may share disassociated search queries with Bing. Microsoft launched its Web IQ grounding service on the Bing index on June 2, 2026, and says it powers Microsoft Copilot and ChatGPT. Seer Interactive's February 6, 2025 study found 87% of ChatGPT search citations matched Bing's top 20 organic results, versus 56% matching Google.
3. Third-party scraping providers
Peec AI's September 4, 2026 report identified three scraping provider labels between May and July 2026: "bright", "oxylabs" and "serp". In July 2026, RESONEO analyzed 1,249 answers: free instant mode pulled 63.6% of results from "labrador" and 34.7% from "bright", while paid thinking mode pulled 74.6% from "bright". On August 21, 2025, The Information reported that OpenAI used SerpApi (a Google results scraper) for some real-time topics; OpenAI has not commented publicly.
4. Licensed publisher partners
As of Search Engine Land's October 21, 2026 update, OpenAI has 18 publisher content partners, including the Associated Press, Axel Springer, the Financial Times, News Corp, Le Monde, Vox Media, The Atlantic, Conde Nast, Hearst, The Guardian, The Washington Post and Reddit.
How to check what ChatGPT knows about your business
Verify crawler access and your indexed pages in three steps to support showing up in ChatGPT:
- Inspect crawler access and your snippet. Run your URLs through our free GEO audit (no account needed, 5 audits a day). It fetches your page as
OAI-SearchBot, checksrobots.txtfor OAI-SearchBot, GPTBot and ChatGPT-User, verifies HTML text without JavaScript, and rebuilds the title and ~200-character snippet ChatGPT keeps. - Check which pages ChatGPT finds and cites. Use FoundByChat (free to start) to ask ChatGPT your customers' questions with web search on, every 2 hours, from the United States or the Netherlands. It records whether your page is "Found" or "Cited" and lists your "Pages in ChatGPT's index" with the stored title and snippet.
- Watch referral traffic. Clicks from ChatGPT carry
utm_source=chatgpt.com, letting you track ChatGPT traffic in GA4.
Common mistakes that hide your site from ChatGPT's sources
OpenAI notes in its October 2026 documentation that ChatGPT ranks search results using multiple factors and its Sources panel can be incomplete, outdated or incorrect. Avoid these mistakes:
- Blocking OAI-SearchBot instead of GPTBot. Disallowing
OAI-SearchBotinrobots.txtremoves your site from ChatGPT search answers. - Assuming ChatGPT-User builds the search index.
ChatGPT-Useronly handles user actions in ChatGPT and Custom GPTs and does not control Search inclusion. - Letting menu or cookie text leak into the snippet of about 200 characters. The GEO audit checks for menu or cookie text leaking into the snippet, text present in the HTML without JavaScript, and facts up front. See ChatGPT citations. FoundByChat emails "Snippet changed" when ChatGPT keeps a new title or snippet and "New in the index" when a page first appears.
FOUNDBYCHAT
Know the moment ChatGPT picks up your pages
We ask ChatGPT your questions every 2 hours and mail you what its index picked up or rewrote. Free to start: sign in with Google or an email link.
Questions
Does ChatGPT get its information from Google or Bing?
ChatGPT search sources include its own index (labrador) and scraping providers. Microsoft says its Web IQ service, launched on June 2, 2026 on the Bing index, powers ChatGPT. Seer Interactive found on February 6, 2025 that 87% of ChatGPT search citations matched Bing's top 20 results, whereas Ahrefs found on August 20, 2025 that 83.39% of ChatGPT's retrieved results did not appear in Google's results at all.
Can ChatGPT read my website if I block GPTBot?
Yes, if you only block GPTBot in robots.txt, ChatGPT can still include your website in live search answers via OAI-SearchBot and fetch pages on user request via ChatGPT-User. OpenAI documents that GPTBot crawls content that may be used to train generative AI foundation models, while OAI-SearchBot is a separate setting that controls whether your pages surface in ChatGPT's search features.
How fast does ChatGPT update when I change my robots.txt file?
According to OpenAI's crawler documentation in October 2026, it can take about 24 hours after a robots.txt change for ChatGPT search to adjust.
Why does ChatGPT show a link to my site even though I blocked OAI-SearchBot?
According to OpenAI's October 2026 publisher FAQ, a URL blocked in robots.txt can still be learned from a third-party search provider and shown as a bare navigational link and page title. To prevent even the bare link and title from appearing, OpenAI states that you must let the crawler read a noindex meta tag on the page.
Sources
- OpenAI Help Center: Searching the web with ChatGPT
- OpenAI Help Center: ChatGPT search for Enterprise and Edu
- OpenAI: Introducing ChatGPT search
- CNBC: OpenAI launches ChatGPT search
- Microsoft Learn: Web IQ FAQ
- Search Engine Land: AI company publisher partnerships
- Search Engine Land: OpenAI used SerpApi for Google search results
- Peec AI: ChatGPT built its own search index
- RESONEO: Study of ChatGPT retrieval sources
- Seer Interactive: 87% of SearchGPT citations match Bing's top results
- Ahrefs: Is ChatGPT really powered by Google?
- Cloudflare: Cloudflare and OpenAI discovery pilot
- OpenAI Developers: Overview of OpenAI crawlers
- OpenAI Help Center: Publishers and developers FAQ
- OpenAI: DevDay 2026 recap
- OpenAI: How people are using ChatGPT
- Common Crawl: CCBot