What you can control
- Crawl access: OAI-SearchBot is OpenAI’s crawler for search; ChatGPT-User fetches pages on behalf of users; GPTBot relates to model training. Allow or block each deliberately in robots.txt.
- Indexing: Bing’s index is widely reported as one source for ChatGPT search, so Bing Webmaster Tools matters.
- Content: pages that answer questions directly and are easy to quote.
Check OpenAI’s current crawler documentation, because names and behaviour change.
What IndexNow does
IndexNow is an open protocol that lets you notify participating search engines, including Bing, when a URL is added, updated or deleted, so they can crawl sooner. It does not guarantee ranking or citation. Many CMS plugins and CDNs support it; for static sites you can submit URLs with a small script.
A set-up sequence
- Verify the site in Bing Webmaster Tools and submit the sitemap.
- Check that robots.txt and your CDN allow OAI-SearchBot and ChatGPT-User.
- Enable IndexNow and ping it when you publish or update.
- Rewrite key pages answer-first with specific facts and sources.
- Track referrals from chatgpt.com in Google Analytics.
- Test 20–30 prompts monthly and record who is cited.
What is uncertain
- How much weight each source gets changes over time.
- Answers vary by user, location and model version.
- Nobody can guarantee a mention, so measure share of voice instead of chasing one prompt.
Check your AI visibility Free review
We run your buyer prompts across ChatGPT, Perplexity, Gemini and Claude and show you the gaps.
Frequently asked questions
Should I block GPTBot?
That is your decision: it concerns training use. Blocking GPTBot does not by itself stop search retrieval via OAI-SearchBot.
Is Bing really that important?
It is a widely reported data source for ChatGPT search and for other AI products, so it is worth ten minutes to verify your site.
Can AWI Digital set this up?
Yes. It is included in our AI visibility and SEO work.
Get a free quote in 24 hours
Tell us what you need. We reply with scope, timeline and a fixed price.