Microsoft Copilot Crawler Checker
Can Microsoft Copilot use your website in its answers? Copilot's web answers come from Bing, so this checker tests Bingbot's access, your robots.txt rules and the page signals Bing relies on.
Check your home page or any page URL. Free, no sign-up, and we don't save results.
- Microsoft's crawlers first
- Live robots.txt check
- Every AI crawler too
- Fix-it advice
Microsoft and Copilot: every crawler at a glance
| User agent | What it does | Follows robots.txt | If you block it |
|---|---|---|---|
| Bingbot | Crawls for Bing, which powers Microsoft Copilot answers and is used as a search source by other AI tools. | Yes | You lose Bing and Copilot visibility, and AI tools that lean on Bing results. |
How Copilot finds websites
Microsoft's documentation says Copilot may fetch information from the Bing search service when web information helps it give a better, more grounded response. In practice, Copilot generates a search query, sends it to Bing and builds its answer from what comes back.
That means your visibility in Copilot's web answers starts with Bing: Bingbot must be able to crawl your pages and Bing must have them indexed. In the Microsoft documentation we reviewed (November 2026) there is no separate robots.txt token for Copilot, so Bingbot access is what this checker tests.
Since October 2025, Bing supports the data-nosnippet attribute. Bing says content marked with it is still indexed normally but is excluded from snippets and AI summaries in Bing Search and Copilot experiences.
Bing also supports IndexNow, which lets you notify Bing the moment a page is added or changed instead of waiting for a recrawl, and Bing Webmaster Tools added an AI Performance report (public preview, February 2026) for tracking how your site appears in AI experiences.
How to allow or block Copilot in robots.txt
Most sites should simply let Bingbot crawl through the User-agent: * group. Don't block Bingbot unless you want to leave Bing and Copilot.
Make sure Bingbot can crawl
Only needed if something else in your robots.txt might restrict it. Remember a named group ignores your * rules.
User-agent: Bingbot
Allow: /
Sitemap: https://www.example.com.au/sitemap.xmlKeep a section out of Copilot answers
Bing indexes the content but excludes it from snippets and AI summaries.
<div data-nosnippet>
This section won't be quoted in Bing or Copilot answers.
</div>How to make sure Bing can find you
Add and verify your site in Bing Webmaster Tools, submit your sitemap and check the index coverage. Many Australian sites only ever set up Google Search Console and have gaps in Bing they don't know about.
To check a visitor claiming to be Bingbot, Bing Webmaster Tools includes a Verify Bingbot tool, and genuine Bingbot requests reverse-resolve to a search.msn.com hostname.
Why Copilot might not see your site
Never checking Bing at all
If your pages aren't in Bing's index, Copilot's web answers can't draw on them. Bing Webmaster Tools shows you what's indexed.
Waiting weeks for recrawls
IndexNow tells Bing about new and updated pages straight away. Many platforms and plugins support it.
A firewall blocking Bingbot
Bot protection that blocks unverified crawlers can catch Bing too if it's misconfigured. Check your security logs for blocked Bingbot requests.
Assuming robots.txt can target Copilot alone
We found no Copilot-specific robots.txt token in Microsoft's documentation. Use data-nosnippet for content you don't want in Copilot answers.
Readable by Copilot is not the same as recommended
Passing this check means Copilot can read your site. Whether it cites or recommends you depends on what it finds: clear answers to your customers' questions, consistent business details and mentions on sites it trusts. That's the work of generative engine optimisation (GEO) and AI SEO, built on solid technical SEO. Our AI search team does exactly this for Australian businesses.
Readable by Copilot is step one. Let's get it recommending you.
Book a free 30-minute AI search strategy session. We'll check where your business shows up, or doesn't, in ChatGPT, Gemini, Perplexity and Google AI Overviews, show you what your competitors are doing, and hand you a written plan to fix it.
- AI visibility review across the major engines
- Competitor analysis: who AI recommends instead of you
- Technical fixes from your crawler check, prioritised
- A written plan that's yours to keep. No obligation.
Apply for your free strategy call
Get your competitor analysis, deep dive and clear report. Takes about 30 seconds, no obligation.
Copilot crawler questions, answered.
Is there a Copilot user agent I can allow or block?
Not in the Microsoft documentation we reviewed in November 2026. Copilot's web answers are grounded in the Bing search service, so Bingbot's access and Bing indexing are what matter.
How do I get my business into Copilot's answers?
Make sure Bing can crawl and index your site (Bing Webmaster Tools, a sitemap, IndexNow), then publish clear, specific content that answers your customers' questions. Being crawlable makes you eligible, but quality and authority decide whether you're used.
How do I keep part of a page out of Copilot?
Wrap it in an element with the data-nosnippet attribute. Bing says that content is still indexed but excluded from snippets and AI summaries in Bing Search and Copilot.
Why doesn't the firewall test run for Bingbot?
Bingbot is verified by its IP addresses, and good firewalls block anyone faking it. A spoofed test would give misleading results, so this checker tests Bingbot through your robots.txt instead.
Sources
- Manage public web access in Copilot (Microsoft Learn)
- Bing introduces support for the data-nosnippet HTML attribute (Bing Webmaster Blog, October 2025)
- Bing Webmaster Blog
Facts on this page were checked against these sources in November 2026. AI crawlers change often, so we review this page regularly.
Check another site, or a competitor's.
See whether Copilot and every other AI crawler can read any website. Free, with no sign-up.
Run the Copilot check