Robots.txt Generator

Build a robots.txt file for your website with allow/disallow rules, crawl-delay and sitemap URL.