llms.txt Generator: Which Tools Actually Work (2026)

Updated October 2026. This page is about the tooling — which generators exist, what each one actually produces, and when to skip them and script it yourself.

What does an llms.txt generator actually do?

All of them follow the same three steps:

1. Crawl. Start from a URL, follow internal links to a configured depth or page cap.
2. Strip. Remove navigation, footers, ads and script, keeping the main content of each page.
3. Emit. Write a Markdown index (llms.txt) and, usually, a concatenated full-text file (llms-full.txt).

What separates a good generator from a bad one is entirely step 3 — specifically whether it writes a one-line description per link and whether it puts the sections in a sensible order. Those two things are what the format exists for, and they're the two things crawling can't infer. A generator that emits 400 bare URLs has produced a sitemap with a different filename.

Which llms.txt generators are worth using?

How do you generate a file with Firecrawl?

Firecrawl's generator at llmstxt.firecrawl.dev is the one worth starting with, because it uses a model to write the per-link descriptions rather than copying page titles. Via the web form: paste a URL, choose the page cap, choose whether you also want the full-text file, generate, download. Via the API:

from firecrawl import FirecrawlApp

firecrawl = FirecrawlApp(api_key="your_api_key")
results = firecrawl.generate_llms_text(
    url="https://example.com",
    max_urls=50,
    show_full_text=True,
)

if results.success:
    print(results.data)
else:
    print(f"Error: {results.error}")

When should you script it instead?

Whenever the file needs to stay current. Crawl-based generators re-derive your structure from HTML every run, which means every run can produce a slightly different file — different ordering, different descriptions, pages that happened to be linked that day. For anything you maintain, a build script reading your own content source is both cheaper and stable.

That's how we do it. The index at public/llms.txt is hand-written — 83 lines, and the descriptions are the entire value. The full-text corpus is scripted: one command pulls all 20 pillar guides from the CMS Admin API, converts each HTML body to clean Markdown, prepends a per-cluster table of contents, and writes public/llms-full.txt idempotently. You can read both outputs at codersera.com/llms.txt and codersera.com/llms-full.txt.

If you're on Astro, Next.js, SvelteKit or anything with a content collection, the same pattern is 30 lines — read the collection, map titles plus descriptions to Markdown list items, write to the static directory at build time.

Why does generator output usually need editing?

Because the measured state of auto-generated files is poor. Common Crawl analysed 584,107 llms.txt files in its July 2026 archive and found:

• 68.3% were template or plugin output rather than curated — Wix alone accounted for 241,000 files.
• 67.1% had no per-link descriptions at all.
• 22.6% contained no links whatsoever.
• 40.4% of successful responses were served as HTML instead of plain text or Markdown.
• Only 49.9% followed the spec's shape (H1, blockquote, H2 link lists).

Median output size by generator shows how thin the plugin tier is: GoDaddy parking pages 148 tokens, Wix 719, Yoast 729, WordPress 1,602, Rank Math 7,459, AIOSEO 8,772.

So a realistic workflow is: generate, then cut. Delete anything you wouldn't want quoted. Rewrite each description to say what the page answers rather than what it's titled. Reorder sections so the first one orients a stranger. Move secondary links under ## Optional. That edit pass takes an hour and it's the entire difference between a useful file and one of the 67%.

Where do you put the generated file?

At your domain root, so it serves from https://yoursite.com/llms.txt — the same place as robots.txt. In practice: public/ in Next.js, static/ in Hugo, Docusaurus and Astro, the web root on a plain static host. Since v2 of the spec you can also publish subpath files such as /docs/llms.txt, which cover the pages beneath that path, with the most specific file winning.

Then verify it, because the most common deployment failure is invisible from the dashboard:

curl -sI https://yoursite.com/llms.txt

You want 200 and a content-type of text/plain or text/markdown. If you see text/html, a framework catch-all route is intercepting the path and every agent that requests the file is receiving a web page instead. docs.cursor.com/llms.txt has exactly this bug today — it redirects to the docs homepage and returns HTML.

Is generating an llms.txt file worth the effort?

Honestly: only if you intend to curate it afterwards, and only if coding agents or AI-native search tools are plausibly part of how people reach you. Server-log evidence is consistent that the major AI crawlers — OpenAI's, Anthropic's, Google's, Perplexity's — do not fetch llms.txt, and Ahrefs found 97% of the files it surveyed received zero requests in a month. The clients that do read them are coding agents and AI-native search startups.

That's not an argument against the file. It's an argument against running a plugin and calling it done. We walk through the full evidence, including 15 days of our own crawler logs, in the complete llms.txt guide.

FAQ

What's the best free llms.txt generator?

Firecrawl's, at llmstxt.firecrawl.dev. It's the only free option that writes real per-link descriptions using a model rather than copying page titles, and it emits both the index and the full-text file. Writesonic's is a reasonable no-signup fallback for a single file.

Is there an llms.txt generator for WordPress?

Several. Yoast SEO, AIOSEO and Rank Math all generate one from installed content, and there's a standalone LLMs.txt Generator plugin on the WordPress directory. Rank Math and AIOSEO produce the most substantial output of the plugin tier; all of them benefit from a manual edit pass.

Do generators also create llms-full.txt?

Firecrawl and Mintlify do. Note that llms-full.txt isn't part of the llms.txt spec at all — it's a community convention — and generated ones get large fast. Keep yours under about 500 KB if you want a model to be able to read it in one pass.

Can I generate llms.txt automatically on every deploy?

Yes, and for any site that changes weekly that's the right approach. Read your own content source rather than crawling your rendered HTML, emit the Markdown at build time, and write it to your static directory. A crawl-based generator re-derives your structure every run and will drift; a build script won't.

Does an llms.txt generator improve SEO?

No. There's no evidence that publishing an llms.txt file improves search rankings or AI citations, and Google has compared it to the deprecated keywords meta tag. Generate one for agent usability, not for rankings.

How often should I regenerate the file?

Regenerate when content changes a section, a number or a recommendation, and always when a page joins or leaves the curated set. Skip it for typos and metadata edits. A stale file that confidently describes last quarter's pricing is worse than no file.

Where should the generated file live?

At the domain root — https://yoursite.com/llms.txt, alongside robots.txt. Verify with curl -sI that it returns 200 with a text/plain or text/markdown content type; text/html means a catch-all route is intercepting it.

Related reading

• llms.txt Examples, Format, and llms-full.txt Explained — the full reference, with a real production file and crawler-log evidence.
• How to create an llms.txt file — the hand-written path, in six steps.
• llm.txt vs llms.txt — which spelling is correct, and why both are everywhere.
• AI coding agents: the complete guide — the clients that actually read these files.