Robots.txt Generator
Build a valid robots.txt in seconds.
User-agent: * Disallow: /admin Disallow: /cart Disallow: /checkout Sitemap: https://example.com/sitemap.xml
How to use the Robots.txt Generator
- 1 Add user-agent rules and disallow paths.
- 2 Add your sitemap URL.
- 3 Copy the file and upload it to your domain root.
Frequently asked questions
Where does robots.txt go?
It must live at the root of your domain, e.g. https://example.com/robots.txt, so crawlers can find it automatically.
Does robots.txt hide a page from Google?
No. Disallow stops crawling but a page can still be indexed if linked elsewhere. Use a noindex meta tag to keep it out of search.
Related tools
SEO
XML Sitemap Generator
Turn a URL list into a valid sitemap.xml.
SEO
Schema Markup Generator
Create JSON-LD structured data.
SEO
Meta Tag Generator
Build SEO meta tags with a live preview.
SEO
URL Slug Generator
SEO-friendly slugs with stop-word removal.
SEO
Robots.txt Tester
Test any path against a live robots.txt, rule by rule.