Free Robots.txt Generator

Generate a proper robots.txt file to control how search engines crawl your website.

Configuration

User-agent

Allow / Disallow Rules

Sitemap URL

Crawl-delay (seconds, optional)

User-agent: *
Disallow: /admin/
Disallow: /private/