# PowerPlatformTip Robots.txt - Optimized for SEO # Updated: 2026-09-07 (ported from the live site's robots.txt, adapted for Chirpy) # Purpose: Guide search engines to valuable content, exclude low-value pages User-agent: * Allow: / # GEO: explicit content-usage signal (see contentsignals.org) # Duplicated into every named bot group below too: per RFC 9309, a crawler # obeys only the most specific group naming its own token and otherwise # ignores "*" entirely - a dedicated GPTBot/ClaudeBot/etc. group never sees # a Content-Signal line that only lives in "*". Content-Signal: search=yes, ai-train=yes, ai-retrieval=yes # Block low-value pages that dilute crawl budget Disallow: /search Disallow: /search/ Disallow: /404.html Disallow: /404 Disallow: /assets/js/ # NOTE: Internal working folders (CONTENT, DOCS, SOCIALMEDIA, GOOGLESEARCHCONSOLE, # NEWOPTIMIZEDPOSTS, SEO reports) are no longer published (excluded in _config.yml) # and now return 404. They are intentionally NOT disallowed here so Googlebot can # re-crawl, see the 404, and drop them from the index ("Crawled - currently not indexed"). # NOTE: /thank-you/ is set to "noindex, follow" in its front matter and is # excluded from the sitemap. It is intentionally NOT disallowed here so # Googlebot can crawl it and honor the noindex directive (a disallowed # page's noindex is never seen, so it lingers in the index). # /tags/ and /page*/ (pagination) are NOT disallowed: they carry