Robots.txt Generator
Builds a robots.txt as copyable plain text in the browser. Enter a user-agent (default *), Allow and Disallow paths, and optional Sitemap URLs. This is an educational builder: it does not claim that Google or Bing will honor the file, and it is not security or legal advice.
Robots.txt generator
Educational builder only. This is not a crawler guarantee, not security advice, and not legal advice. Google, Bing, and other crawlers are not promised to honor this file. It does not block attackers, hide pages from people, or act as a firewall.
Client-side only. Values are not sent to a server. Output is copyable plain text. There is no /robots-generator/ or /robots-txt/ URL.
How to use
- Keep or change User-agent (default *). Add at least one Allow or Disallow path, one per line, starting with / or *.
- Optionally add Sitemap URLs that start with http:// or https://.
- Read the robots.txt as text and copy it on this device. This page does not upload the file or ask crawlers to fetch it. Refreshing the page clears the form.
Results
Enter at least one Allow or Disallow path to build a robots.txt.
Runs in your browser. Nothing is uploaded.
Frequently asked questions
What does this robots.txt generator produce?
It builds copyable robots.txt text in your browser from a user-agent, Allow paths, Disallow paths, and optional Sitemap URLs. The output is a plain-text string you can copy. This is an educational builder, not a crawler, not a hosting panel, and not a way to publish a file to your site.
Will Google or Bing honor this file?
This page does not claim that Google, Bing, or any other crawler will honor the file. robots.txt is a request that some well-behaved crawlers may read. It is not a guarantee of crawl, index, or ranking behavior. Place the file on your own host if you want crawlers to be able to fetch it; this page does not upload it.
Does robots.txt block attackers or hide pages from hackers?
No. robots.txt is not a firewall, not access control, and not security advice. It does not hide pages from people, hide URLs from attackers, or replace authentication. Anyone can read a public robots.txt. Do not treat this file as legal advice or as a way to keep content private.
Which lines can I add?
This builder writes User-agent, Allow, Disallow, and optional Sitemap lines only. User-agent is required after trim (default *). Paths must start with / or *. Sitemap values must be valid http:// or https:// URLs. An empty group with no Allow and no Disallow is rejected. Line order is User-agent, then Disallow, then Allow, then Sitemap.
Is this client-side? Do I need an account?
Yes, it is client-side only. There is no signup. User-agent, paths, and sitemap URLs never leave the browser. Tallymill does not save what you enter and does not fetch your site. Refreshing the page clears the form.
Is there a size limit?
Yes. Any field or the whole paste larger than 100KB (100 × 1024 UTF-8 bytes) is rejected. Empty or whitespace-only user-agent, paths that do not start with / or *, and non-http(s) sitemap values also fail closed. Nothing is sent to a server either way.