# OAI-SearchBot

> Source: https://rankxai.com/glossary/oai-searchbot · Last updated: 2026-08-18

OAI-SearchBot is the crawler that builds the index behind ChatGPT search. OpenAI documents it separately from GPTBot, which collects training data, and from ChatGPT-User, which fetches a page because somebody asked. Disallowing OAI-SearchBot in robots.txt is what removes a site from ChatGPT search results.

## What does OAI-SearchBot do that GPTBot does not?

OAI-SearchBot builds a searchable index; [GPTBot](/glossary/gptbot) builds a training corpus. That is the whole distinction, and it decides which of the two you can afford to block. Content OAI-SearchBot has indexed can be surfaced and linked in a ChatGPT answer today. Content GPTBot has collected may influence a model released in a year, with no link and no attribution.

OpenAI publishes the user agent as `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot`. Note that it opens with an ordinary Chrome string: a filter written to match “bot” or “compatible” at the start of a user agent will miss it entirely.

## Why do sites block OAI-SearchBot by accident?

Because the instruction people mean to give is “keep my content out of AI training”, and the rule they write is a blanket one. Two patterns do it. The first is a wildcard group that disallows everything and then allows a named list written before ChatGPT search existed. The second is a content delivery network setting labelled “block AI bots”, which is one switch over a category the vendor defines rather than a per-purpose choice you made.

The result is a site absent from ChatGPT search results whose owner believes it has only opted out of training. It is worth checking rather than assuming, because nothing reports it: there is no console, no warning, and no traffic to notice missing, since a citation that never happened leaves no trace.

## How do you confirm a request really came from OAI-SearchBot?

By address, never by user agent. A user agent string is a request header, so anything can send `OAI-SearchBot/1.4` and plenty of things do, including scrapers hoping a permissive rule applies to them. OpenAI publishes the address ranges its crawlers use as JSON, one file per bot:

- `openai.com/searchbot.json` for OAI-SearchBot
- `openai.com/gptbot.json` for GPTBot
- `openai.com/chatgpt-user.json` for ChatGPT-User

Match the source address of the request against the file for the bot it claims to be. Anything that fails is not OpenAI, whatever its headers say. The honest limit: this tells you a request was genuine, and it tells you nothing about whether the page was used, because OpenAI publishes no feedback of any kind about what its index retained. Nothing in this verification path tells you frequency either, so a site seeing one OAI-SearchBot request a month and a site seeing a thousand have no way to know which of them is normal.

## Sources

- [OpenAI bot documentation](https://developers.openai.com/api/docs/bots), checked 2026-08-18