Robots.txt Tester tells you whether a URL path is allowed or blocked by a robots.txt file. Paste the file contents, enter a path such as /admin/page, choose a user agent, and it reports the verdict and the rule that decided it.
Matching follows the rules Google documents: the most specific user-agent group applies, the longest matching path wins, and Allow wins a tie. The * wildcard and the $ end-of-URL anchor are supported. Test changes here before uploading them, because a mistaken Disallow can remove a whole section from search.
Advertisement
How to use Robots.txt Tester
1
Paste your robots.txt
The whole file.
2
Enter a path and user agent
For example /blog/post and Googlebot.
3
Read the result
Allowed or blocked, with the matching rule.
Use Cases
- Confirming a new Disallow does not block key pages
- Debugging a page reported as blocked in Search Console
- Checking wildcard rules before deployment
Frequently Asked Questions
The one with the longer path. If they are the same length, Allow wins.
No. A crawler follows only the most specific group that names it.
No. Paste the contents so you can also test a draft.
Yes, Robots.txt Tester is 100% free with no usage limits, no watermarks, and no hidden charges. The platform is supported by non-intrusive advertising so all tools stay free for everyone.
Yes. Robots.txt Tester runs entirely in your browser using client-side JavaScript. Your files and data never leave your device and are never transmitted to our servers.