Our bot
CitedCheck
CitedCheck visits a site only when someone asks Cited to check it. It is not a crawler: it never follows links on its own and never runs on a schedule.
User agent: Mozilla/5.0 (compatible; CitedCheck/1.0; +https://citedfix.pages.dev/bot/)
- Free check: the homepage, robots.txt, llms.txt, the sitemap and, for Shopify stores, the public product feed and one product page. Around six requests.
- Paid audit: up to 10 pages of the site, at most one request at a time.
- It fetches only public pages over HTTP(S), never logs in or submits forms, and refuses private network addresses.
- Firewall test: to see whether your firewall turns AI search crawlers away, we request the homepage once more with a user agent that imitates OpenAI’s search crawler (OAI-SearchBot), reading at most 20 KB. It comes from our Cloudflare Worker, not from OpenAI, so it won’t pass any verification of OpenAI’s real IP addresses; that’s fine for the test.
- robots.txt rules for AI crawlers are read from your file, not tested by crawling.
Blocking it
Add User-agent: CitedCheck with Disallow: / to robots.txt, or block the user agent at your firewall. Questions or problems: contact@optimumtechfusion.com.