How to rank in ChatGPT: what we measured on a real site

How to rank in ChatGPT starts with access, not content. ChatGPT search can only show pages its search crawler, OAI-SearchBot, is allowed to fetch. So allow OAI-SearchBot in robots.txt, confirm your server actually returns a 200 to it rather than a 403, then write pages with short, direct answers a model can lift and cite. Nobody can promise the citation itself.

What we found when we tested a real site

On one SEO Autopilot engagement, a US healthcare analytics company on WordPress behind a firewall, the first thing we found had nothing to do with content. The firewall returned 403 to OpenAI's and Anthropic's search crawlers on every page.

The trigger was a generic bad-bot rule matching the substring "searchbot". OpenAI's search crawler is called OAI-SearchBot, so the rule caught it by name. Eleven other crawlers passed. The client's monitoring tool reported all sixteen AI crawlers as accessible, because it read robots rules instead of sending requests.

That gap is the whole lesson of this article. robots.txt says what you intend. The firewall decides what happens. If you want to know how to rank in ChatGPT, the first question is not about keywords or content; it is whether OpenAI's search crawler gets a page or an error when it asks.

Over the engagement, which included removing that rule, the site's AI Visibility score went from 18.35 to 66.35 on a rubric frozen at baseline. The detail, and what got worse, is further down.

Can you rank in ChatGPT if OAI-SearchBot is blocked?

No, not in ChatGPT search answers. OpenAI's overview of its crawlers says OAI-SearchBot is used to surface websites in search results in ChatGPT's search features, and that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links.

The same page describes two other agents, and it helps to keep them apart because they do different jobs:

OpenAI states that each setting is independent of the others: a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot. So if your worry is training, you do not have to give up ChatGPT search to act on it. For SEO for ChatGPT, OAI-SearchBot is the one that matters.

How to rank in ChatGPT: the steps we follow

These are the steps we run on a client site, in this order. The first three are about access; skipping them makes everything after them guesswork.

1. Allow OAI-SearchBot in robots.txt. OpenAI recommends allowing it in your robots.txt file and allowing requests from its published IP ranges. Check that no broad Disallow or wildcard group catches it.
2. Check your firewall, CDN and security plugin rules for anything matching crawler names or substrings such as "bot" or "searchbot". This is where our client's block lived, and robots.txt never showed it.
3. Confirm the live response. Request your pages with the OAI-SearchBot user agent and read the status code. A 200 is what you want; a 403 means you are refused.
4. Give it time. OpenAI notes it can take about 24 hours from a robots.txt update for its systems to adjust.
5. Fix the page basics: one H1 per page, a clear title, and a sitemap that lists the pages you want found.
6. Rewrite your key pages so each section opens with a direct answer to the question a buyer would type.
7. Re-measure against the same baseline, so you can see what moved and what did not.

The page basics in step five are not decoration. On the same healthcare analytics site, 21 of 39 pages had no H1 because the post template rendered the title as an H2. One template change took pages with exactly one H1 from 9 of 39 to 30.

How do you confirm OAI-SearchBot gets a 200, not a 403?

Send a request that looks like the crawler's and read the status code, rather than reading your rules and assuming. A rules check and a request check can disagree, and on our client's site they did.

A 403 is not a temporary hiccup. MDN's reference for 403 Forbidden says the server understood the request but refused to process it, and that clients receiving a 403 should expect that repeating the request without modification will fail with the same error. In other words, a crawler refused today will be refused tomorrow until someone changes the rule.

If your site sits behind Cloudflare, rule order matters. Cloudflare's custom rules documentation says custom rules are evaluated in order, and that some actions such as Block stop the evaluation of other rules, so a later rule will not run for a request an earlier rule blocked. The same page says the Skip action can be used to skip one or more Cloudflare security features. An allow rule placed below a broad bot block may never be reached.

Our free AI Visibility Checker is a reasonable first check. It reads robots.txt rules for 11 AI crawlers and requests your homepage once with each crawler's user agent. That is the checker's own request, not the real crawler's, and it covers the homepage only. A firewall can treat OpenAI's real traffic differently from ours, and an inner template can behave differently from the homepage. Treat a pass as a good sign, not proof that ChatGPT can reach every page.

How to get cited by ChatGPT once the crawler gets through

Write pages that answer the question in the first two sentences of each section, in plain words, then add the detail. That is our practice, not a documented OpenAI rule, and nobody outside OpenAI can promise a citation.

Our reasoning is simple. An answer engine assembles a reply from passages. A section that opens with a self-contained answer is easier to lift than one that warms up for three paragraphs first. What we do on client pages:

• Phrase at least some headings as the question a reader would type.
• Open each section with a one or two sentence answer that makes sense out of context.
• Put numbers, steps and comparisons in lists and tables rather than long paragraphs.
• Say who wrote the page and what they have actually done, and cite sources for claims you did not measure yourself.
• Keep one topic per page, so the page has a clear answer to be cited for.

Do not drop ordinary SEO while you do this. For Google's own AI features, Google's guide to optimising for generative AI features says SEO best practices continue to be relevant because those features are rooted in its core Search ranking and quality systems, and that a page must be indexed and eligible to be shown with a snippet. That guide is about Google, not ChatGPT, but the habits it rewards, crawlable and indexable pages with clear answers, are the same ones we build for ChatGPT SEO. For the wider picture, our piece on LLM SEO covers the other answer engines.

What we have seen: the numbers from one engagement

Baseline frozen 15 July 2026, re-measured 18-19 August 2026 by fetching all 39 sitemap URLs. These are scores on our own rubric, not ChatGPT traffic or citation counts.

Two rows are worth reading carefully. Performance fell from 48 to 28, because removing the firewall rule took the CDN edge cache with it: median time-to-first-byte went from 0.42s to 2.39s. Unblocking a crawler can cost you speed if the block and the cache are tied together, so check both. AI review signals did not move at all, from 18 to 18; access and markup fixes do nothing for what others say about you.

In the AI Visibility score, AI crawler access carries 20% of the weight and AI schema and structured data 25%, which is why access and markup fixes weigh so heavily in the total. The full account of the firewall finding is in our write-up of the firewall that blocked AI crawlers.

Is ChatGPT SEO a separate job from Google SEO?

Partly. The access layer is separate, because ChatGPT search depends on OpenAI's crawler and Google depends on its own. The content layer overlaps heavily: clear pages with direct answers serve both.

In practice that means checking two things on a schedule. First, that OAI-SearchBot and the other search crawlers you care about still get a 200, because a plugin update or a new firewall rule can undo it silently. Second, that new pages are written for extraction from the start. That is the work SEO Autopilot does every month for one site, at ₹20,000 a month (about US$230), with no setup fee and no minimum term.

Frequently Asked Questions