Create a robots.txt file without the guesswork.
Pick the paths you want crawlers to skip, add your sitemap and copy the finished file. It runs in your browser and stores nothing.
Generate your robots.txt
Edit the options and watch the file update. Then upload it to the root of your site.
What the generator covers.
The basics of a safe, simple robots.txt file.
Paths to block
List folders you do not want crawled, such as admin or cart pages.
- One path per line
- Adds missing slash
- Warns about blocking everything
Sitemap line
Adds your sitemap URL so crawlers can find it quickly.
- Sitemap URL
- Absolute address
- Optional
AI crawlers
Optionally block crawlers used for AI training. Blocking them is your choice.
- GPTBot and ClaudeBot
- Google-Extended
- PerplexityBot and CCBot
Three steps to useful results.
Add your paths
Type the folders you want crawlers to skip. Leave it empty to allow everything.
Add your sitemap
Paste your sitemap URL, then choose any AI crawlers you want to block.
Copy and upload
Save the text as robots.txt in your site root, then test it with our Robots.txt Tester.
Robots.txt Generator FAQ
Common questions about this tool and what the results mean.
Ask us about your resultsDoes robots.txt hide a page from Google?
No. It asks crawlers not to crawl a page. To keep a page out of search, use a noindex tag or a password. Read our robots.txt guide.
Will blocking AI crawlers remove me from Google Search?
No. Google-Extended only controls use for Google AI training and does not affect Search ranking. Blocking other AI crawlers may reduce how often AI tools see your pages.
Where does robots.txt go?
In the root of your domain, for example seovoro.com/robots.txt.
Does the tool store what I type?
No. It runs in your browser and sends nothing to our server.
Want an expert to fix what the tool finds?
Our team turns diagnostics into a prioritized roadmap and does the work. Start with a free audit request.