GUIDE

LLM SEO guide to training data and live retrieval

In FoundByChat's October 2026 study, position 1 in ChatGPT's first search was cited 55.5% of the time, with 2.2 URLs cited per answer.

Updated October 30, 20265 min read

55.5%Citation rate for position 1 in ChatGPT's first search (FoundByChat, Oct 2026)
63.6%Free instant ChatGPT search results from OpenAI's index (RESONEO, Jul 2026)
10.3%Share of retrieved web results that ChatGPT cites (FoundByChat, Oct 2026)

In short

LLM SEO and ChatGPT SEO involve foundation model training-related crawling (GPTBot, CCBot) and search-related crawling (OAI-SearchBot, PerplexityBot).
Practical wins come from live retrieval, where ChatGPT cites an average of 2.2 URLs per answer (FoundByChat's October 2026 study of 218 answers).
OpenAI's crawlers are independent: blocking GPTBot opts out of training without hurting ChatGPT search visibility if OAI-SearchBot is allowed.
Citation rates varied by search position in the study: position 1 in ChatGPT's first search is cited 55.5% of the time, versus 0.4% at position 16 and lower.

Step 1: Separate training bots from search and user bots in robots.txt

AI companies run separate bots for training, search indexing and user requests. Configure robots.txt and your firewall so search and user bots can reach your pages, even if you block training crawlers.

Provider	Search indexing control	Training data control	User-action fetcher
OpenAI	`OAI-SearchBot`	`GPTBot`	`ChatGPT-User`
Anthropic	`Claude-SearchBot`	`ClaudeBot`	`Claude-User`
Perplexity	`PerplexityBot`	Not used for foundation models	`Perplexity-User`
Google	`Googlebot`	`Google-Extended` (token)	Not published
Apple	`Applebot`	`Applebot-Extended` (token)	Not published
Microsoft	`Bingbot` (`NOARCHIVE` / `NOCACHE` meta tags control Copilot answers)	Not published	Not published
Mistral	Not specified in this fact sheet	Not specified in this fact sheet	`MistralAI-User`

Per OpenAI's bot documentation (retrieved October 29, 2026), OAI-SearchBot surfaces sites in ChatGPT search, while GPTBot crawls content for generative AI foundation model training. Disallowing GPTBot tells OpenAI the content should not be used for training; allowing OAI-SearchBot supports eligibility for ChatGPT search but does not guarantee placement. ChatGPT-User handles user actions in ChatGPT and Custom GPTs, does not affect search inclusion, and may ignore robots.txt. For ChatGPT ads, OpenAI requires OAI-AdsBot and recommends OAI-SearchBot.

Other providers document similar rules in 2026:

Anthropic and Perplexity: ClaudeBot collects web content that may be used for training, Claude-SearchBot supports search, and Claude-User handles user requests. PerplexityBot indexes for search without training foundation models; Perplexity-User fetches pages for users and generally ignores robots.txt.
Google, Apple and Microsoft: robots.txt rules for Googlebot control how a site appears in Search, including AI Overviews and AI Mode, while Google-Extended controls Gemini training and Gemini Apps grounding without affecting Search rank. Applebot powers Spotlight, Siri and Safari search; Applebot-Extended opts out of training. Bing and Copilot share Bingbot (NOARCHIVE blocks Copilot answers; NOCACHE limits them to URL, title and snippet).

Step 2: Optimize for the search indexes LLMs actually query

When ChatGPT search partners with other search providers, it typically rewrites a query into one or more targeted queries for those providers; Ahrefs found in August 2025 across 118,931 fan-out queries that ChatGPT ran 1.78 searches per prompt on average.

To understand where ChatGPT gets its info, look at its retrieval backends:

OpenAI's own index ("labrador"): Between May and July 2026, Peec AI saw ChatGPT label result sources as labrador (OpenAI's index) or scrapers bright, oxylabs and serp. RESONEO's July 2026 analysis of 1,249 answers found free instant mode pulled 63.6% of results from labrador and 34.7% from bright, while paid thinking mode pulled 74.6% from bright. Through a July 2026 Cloudflare discovery pilot, OpenAI also indexed hundreds of millions of new pages by October 15, 2026.
Bing and Microsoft Web IQ: At ChatGPT search's October 2024 launch, OpenAI's Srinivas Narayanan called Bing "an important one", and OpenAI notes Enterprise and Edu searches may share disassociated queries with Bing. On June 2, 2026, Microsoft launched Web IQ on the Bing index, stating it powers Copilot and ChatGPT. In February 2025, Seer Interactive found 87% of ChatGPT search citations matched Bing's top 20 organic results, versus 56% for Google.
Google and publisher partnerships: Ahrefs (August 2025) found only 6.82% of ChatGPT's retrieved results were in Google's top 10, and 83.39% missed Google entirely. The Information reported in August 2025 that OpenAI used Google scraper SerpApi for some real-time topics (unconfirmed by OpenAI). Search Engine Land (October 2026) notes OpenAI has 18 publisher partners, including AP, the Financial Times, News Corp, Vox Media, Conde Nast, The Guardian, The Washington Post and Reddit.

Step 3: Put facts up front and check the snippet

Earning ChatGPT citations requires high retrieval rank and a clean snippet. In FoundByChat's October 2026 study of 218 answers, ChatGPT cited just 10.3% of retrieved results (Reddit was 28% of retrieved results but got only 2 of 2,462 citations).

Citation rates by position in ChatGPT's first search in the study:

Position 1: cited 55.5% of the time (95% CI 49.1% to 61.9%)
Position 2: cited 34.9% of the time
Position 3: cited 27.5% of the time
Positions 4 and 5: cited 14.4% of the time
Positions 6 to 10: cited 8.1% of the time
Positions 11 to 15: cited 2.4% of the time
Position 16 and lower: cited 0.4% of the time

Run a free GEO audit (no account, up to 5/day) to fetch the page using the OAI-SearchBot user agent. It rebuilds the title and ~200-character snippet ChatGPT keeps, looks the page up in ChatGPT, and scores your URL out of 100 across non-JS HTML text, title/H1, menu or cookie text in the snippet, facts up front, JSON-LD, llms.txt, robots.txt, noindex and sitemap.

Step 4: Measure LLM visibility and avoid common mistakes

ChatGPT referral clicks carry utm_source=chatgpt.com, but analytics alone may miss when a page is found but not cited; in FoundByChat's study, ChatGPT cited 10.3% of retrieved web results.

With FoundByChat, you add your site and customer questions, and the app runs them in ChatGPT with web search every 2 hours from the US (English) or the Netherlands (Dutch). It tracks whether your page was "Found", "Cited", its usual position, and your "Pages in ChatGPT's index" with the exact title and snippet kept. It is free to start (no password or card) and emails you on "New in the index", "Snippet changed" or "No search results".

When comparing GEO vs SEO, avoid three mistakes:

Using robots.txt to hide a URL: Blocked URLs can still show as bare links via third-party search providers; use a readable noindex tag instead.
Letting cookie banners or JS hide key text: The audit checks for menu or cookie text leaking into ChatGPT's approximately 200-character snippet.
Blocking OpenAI IPs in your firewall: Allowing OAI-SearchBot fails if your WAF blocks the IPs at openai.com/searchbot.json.

FOUNDBYCHAT

Know the moment ChatGPT picks up your pages

We ask ChatGPT your questions every 2 hours and mail you what its index picked up or rewrote. Free to start: sign in with Google or an email link.

Start free Or run the free GEO audit

Questions

What is the difference between LLM SEO, GEO and AEO?

LLM SEO, Generative Engine Optimization (GEO) and Answer Engine Optimization (AEO) all describe preparing web pages so AI assistants find, quote and link them. For a breakdown of the terminology, see our guide on what is AI SEO. In practice, all three rely on allowing search crawlers like OAI-SearchBot and putting clear, factual answers in your HTML.

Does blocking GPTBot hurt my visibility in ChatGPT search?

No. Blocking GPTBot only opts your content out of training foundation models; OAI-SearchBot controls ChatGPT search inclusion independently.

How long does it take for robots.txt changes to affect AI search?

According to official crawler documentation retrieved in October 2026, OpenAI (OAI-SearchBot) can take about 24 hours and Perplexity (PerplexityBot) can take up to 24 hours to adjust after a robots.txt change. DuckDuckGo states that an opt-out for DuckAssistBot takes effect after 72 hours, and Amazon caches robots.txt rules for Amazonbot for up to 30 days.

Why does my page show as a bare link in ChatGPT even though I blocked OAI-SearchBot?

OpenAI's publisher FAQ states that if you block a page in robots.txt, ChatGPT search can still learn about the URL from a third-party search provider and display it as a bare link and page title. To remove it completely, allow the crawler to read a noindex tag on the page.

What does FoundByChat track?

FoundByChat checks ChatGPT search only, recording when your pages are found in search results, cited in answers, or when ChatGPT keeps a new title or snippet.