Tool
Verse
All Tools
SEO
Image
Text
Dev
Calculators
Converters
Advertisement Banner (728×90)
Home
Robots.txt Generator
Robots.txt Generator
Create a proper robots.txt file to control crawler access.
Default Crawlers Rule
Allow all crawlers
Block all crawlers
Disallow Paths (one per line)
Allow Paths (one per line)
Block Specific Crawlers
ChatGPT (GPTBot)
Common Crawl
Claude (Anthropic)
Google Bard
Facebook
Twitter/X
Ahrefs
Semrush
Majestic
Moz
Sitemap URL (optional)
Generated robots.txt
Copy
Download
Advertisement
(300×250)
Popular Tools
Free Open Graph Tag Generator
Free Keyword Density Checker
Free Domain Authority Checker
Free Backlink Checker
Free Text Diff Checker
Free Plagiarism Checker
Categories
Free Text Tools
Free SEO Tools
Free Image Tools
Free Developer Tools
Free Calculator Tools
Free Converter Tools
Advertisement
(300×200)