llms.txt generator changelog
Notes for llms.txt generator only.
llms.txt generator
Störtebeker:Guides on how the generator works and on real llms.txt files
There are two guides on the site now. One explains how the free generator builds a file. The other shows how real sites open their llms.txt, with a small template to start from.
Störtebeker:The generator only fetches public sites
FixedThe generator now checks where an address really points before it fetches anything, so it can't be used to reach private networks. There's also a limit on how often one visitor can run it.
Störtebeker:Better llms.txt files for bigger sites
FixedThe generator now spreads a site's pages across sections and picks up pages that are already Markdown. The validator also stopped flagging plain list items that are perfectly valid.
Störtebeker:llms.txt generator is on the public web
MilestoneThe llms.txt generator is live. Give it a site and it builds and checks an llms.txt file from the public sitemap and robots.txt, or an AGENTS.md for a GitHub repository. It stores nothing and uses no AI model.