GUIDE

How to do AI search optimization across every engine

BrightEdge found ChatGPT sent 95.1% of AI referral traffic in August 2026, while Google AI Overviews passed 2.5 billion monthly users.

Updated October 30, 20265 min read

95.1%Share of AI referral traffic sent by ChatGPT, BrightEdge Aug 2026
2.5B+Monthly users of Google AI Overviews, Google May 2026
43%AI citation odds at Google rank 1 vs 5% at rank 7, SSRN Feb 2026

In short

Allow search crawlers (OAI-SearchBot, Googlebot, PerplexityBot, Bingbot) in robots.txt while choosing separately whether to block training crawlers and tokens (GPTBot, Google-Extended, ClaudeBot).
A December 2024 Vercel and MERJ study found that AI crawlers from OpenAI, Anthropic, Meta, ByteDance and Perplexity do not render JavaScript, so put core facts in server-sent HTML.
A February 20, 2026 SSRN study of 1,736 pages found Google rank predicted AI citations (43% at rank 1 vs. 5% at rank 7), while Schema.org presence did not.
Ahrefs (September 2026, 115,254 sites) measured AI traffic at 0.43% vs. 26.82% from search; Adobe found AI visitors to U.S. retail sites converted 90% better than non-AI visitors in September 2026.

This checklist covers AI search engine optimization across ChatGPT, Google AI Overviews, AI Mode, Perplexity and Microsoft Copilot.

Step 1: Allow the right search crawlers in robots.txt

ChatGPT, Perplexity and Copilot use OAI-SearchBot, PerplexityBot and Bingbot respectively for search. GPTBot is OpenAI's training crawler. Blocking OAI-SearchBot keeps pages out of ChatGPT search answers, though they can still appear as navigational links.

OpenAI documents that OAI-SearchBot surfaces websites in ChatGPT search results, while GPTBot crawls content that may be used to train generative AI foundation models. The settings are independent: blocking GPTBot does not remove your site from ChatGPT search, and blocking OAI-SearchBot keeps your pages out of ChatGPT search answers (though they can still appear as navigational links). See our guide on how ChatGPT search works.

Engine	Search and index control	Training control	Live user fetch bot
ChatGPT	OAI-SearchBot	GPTBot	ChatGPT-User (robots.txt may not apply)
Google AI Overviews & AI Mode	Googlebot	Google-Extended (token only)	Not specified
Perplexity	PerplexityBot (robots.txt changes can take up to 24 hours)	Not specified; PerplexityBot is not for training	Perplexity-User (generally ignores robots.txt)
Microsoft Copilot	Bingbot (plus NOARCHIVE / NOCACHE tags)	Not specified	Not specified
Claude	Claude-SearchBot	ClaudeBot	Claude-User (honors robots.txt)
Apple (Siri, Spotlight, Safari)	Applebot (follows Googlebot rules if robots.txt names Googlebot but not Applebot)	Applebot-Extended (token only)	Not specified

Check meta tags alongside robots.txt. OpenAI's publisher FAQ states that a URL blocked in robots.txt but learned from a third-party search provider may still show in ChatGPT as a bare link and page title, unless a crawler-readable noindex tag is present. Microsoft documents that Bing and Copilot share Bingbot, where NOARCHIVE keeps content out of Copilot answers and NOCACHE limits Copilot to the URL, title and snippet.

Step 2: Serve your main text in HTML without JavaScript

In December 2024, Vercel and MERJ found that the major AI crawlers they studied did not render JavaScript, including OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, and crawlers from Perplexity. Text that requires client-side JavaScript to appear would not be present to those crawlers without rendering.

In December 2024, Vercel and MERJ found that OAI-SearchBot, ChatGPT-User, GPTBot, ClaudeBot, Perplexity and bots from Meta and ByteDance download .js files without running them (11.50% of ChatGPT requests and 23.84% of Claude requests were for JavaScript files). Google's Gemini uses Googlebot's infrastructure, which renders JavaScript, and Applebot renders pages in a browser.

To check what ChatGPT's search crawler receives on your pages, run our free GEO audit (up to 5 audits a day, no account needed). It fetches your URL with the OAI-SearchBot user agent, verifies that text is in the HTML without JavaScript, and rebuilds the title and ~200-character snippet that ChatGPT's search keeps.

Step 3: Put direct answers, plain facts and tables up front

Google reported on October 28, 2026 that AI Mode queries are 3 times longer than traditional searches, and that Google AI Mode passed 1 billion monthly users in May 2026.

Match your title and H1 to the question, and answer it in sentence one. Keep cookie banners and menus out of the opening text so they do not leak into the snippet of about 200 characters that ChatGPT's search keeps.
State specific numbers, dates, prices and specifications in plain prose and tables.
Keep traditional search fundamentals strong. The February 2026 SSRN study showed that Google organic rank was the strongest predictor of AI citation. See GEO vs SEO.

Step 4: Stop counting on Schema.org markup to win AI citations

Google's AI features documentation states that no special Schema.org structured data is needed to appear in AI Overviews or AI Mode; a page must be indexed and eligible for a snippet. While structured data helps with classic rich results, the following studies and test provide evidence about markup and AI search:

Ahrefs: Compared 1,885 pages that added JSON-LD against 4,000 matched controls. AI Overview citations changed by -4.6%, AI Mode by +2.4% and ChatGPT by +2.2%, with the last two not statistically different from zero.
Roskilde University (2026): A randomized controlled trial using 8 synthetic brands (4 with Schema.org markup) found no evidence that markup improved visibility in Gemini with Google Search grounding.
Mark Williams-Cook (May 28, 2026): Hid a fake address in invalid JSON-LD. Both ChatGPT and Perplexity returned it, showing they read the block as plain HTML text rather than parsing Schema.org data.

Put every fact you want cited into visible body text. See our ChatGPT citations guide.

Step 5: Track AI search referrals and index changes per engine

In your analytics platform, segment referrals by source. OpenAI documents that ChatGPT search clicks carry utm_source=chatgpt.com. BrightEdge reported in September 2026 that ChatGPT sent 95.1% of AI referral traffic in August 2026 (Gemini 2.4%, Perplexity 1.1%, Claude 0.8%), growing 101% from January to August 2026. Adobe reported in October 2026 that, in September 2026, AI visitors to U.S. retail sites brought 83% more revenue per visit than non-AI visitors.

FoundByChat checks ChatGPT search directly: add your website and customer questions, and FoundByChat asks ChatGPT with web search on every 2 hours as a user in the US (English) or the Netherlands (Dutch). It records whether your page was "Found" or "Cited", lists your "Pages in ChatGPT's index" with the title and snippet ChatGPT keeps, and emails you on "New in the index", "Snippet changed" or "No search results". It is free to start and checks ChatGPT search only.

FOUNDBYCHAT

Know the moment ChatGPT picks up your pages

We ask ChatGPT your questions every 2 hours and mail you what its index picked up or rewrote. Free to start: sign in with Google or an email link.

Start free Or run the free GEO audit

Questions

Does blocking GPTBot stop my website from appearing in ChatGPT search?

No. Blocking GPTBot only tells OpenAI not to use your content to train generative AI foundation models. ChatGPT search relies on OAI-SearchBot. When OAI-SearchBot is allowed in robots.txt, your pages can appear in ChatGPT search answers.

Do I need special Schema.org markup for Google AI Overviews or ChatGPT?

Google says no special Schema.org structured data is required for AI Overviews or AI Mode. OpenAI has made no statement that ChatGPT parses JSON-LD. In Ahrefs' study, the AI Mode and ChatGPT citation changes were not statistically different from zero; the Roskilde trial found no evidence of improved Gemini visibility.

How long do robots.txt changes take to update in AI search engines?

OpenAI documents that after a robots.txt change it can take about 24 hours for ChatGPT search to adjust, and Perplexity documents that robots.txt changes for PerplexityBot can take up to 24 hours. DuckDuckGo states that a DuckAssistBot opt-out takes 72 hours, and Amazon documents that Amazonbot caches robots.txt rules for up to 30 days.

Is AI search replacing Google Search traffic?

No. Although Gartner predicted in February 2024 that traditional search volume would drop 25% by 2026, Google handles over 5 trillion searches a year (as of March 2025), reported in Q3 2026 its highest year-over-year query growth rate in five years, and grew Search and other revenue 17% year over year in Q2 2026. See our guide on whether SEO is dead.

How can I check what ChatGPT's search crawler keeps from my page?

Run a free GEO audit without an account, up to 5 audits a day. It fetches your page with the OAI-SearchBot user agent and rebuilds the title and the snippet of about 200 characters that ChatGPT's search keeps. To monitor customer questions every 2 hours, connect your site to FoundByChat.