Skip to content

Site-wide checks

These checks apply to the whole site and run on every check, whatever pages you monitor.

Detects any change to robots.txt. If the file starts blocking the entire site from crawling, a critical finding is raised.

Shows what robots.txt says to AI and search crawlers, including GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot and other supported agents, and records when those rules change.

SEO Sentry reports these rules; it never modifies them. See also AI crawler tracking.

Checks that the XML sitemap is reachable and valid, and detects significant drops in the number of URLs it lists.

Detects when Settings → Reading → Discourage search engines from indexing this site is enabled or changed.