
Upload the file to the root of your domain, e.g. https://example.com/robots.txt.
/admin/ or /*?sort=, and exceptions under "Allowed paths".robots.txt and upload it to the root of your domain.User-agent: * Disallow: /admin/ User-agent: GPTBot User-agent: CCBot Disallow: / Sitemap: https://example.com/sitemap.xml
Does robots.txt hide a page from Google? No. It only stops crawling; a blocked URL can still be indexed if other sites link to it. Use a noindex meta tag to keep a page out of search results.
If I add a rule for Googlebot, does it still follow the * rules? No. A crawler uses only the most specific group that matches it, so the tool copies your blocked paths into the group of allowed bots.
Does blocking AI crawlers affect my Google ranking? No. Google-Extended only controls use in Gemini; Googlebot still crawls your site for search.