Generate an llms.txt-style file from sitemaps

What this workflow is for

Use this when your source material already exists but is scattered across pages, notes, docs, or exports.

Riflet helps collect and structure the material. You still decide what belongs in the public llms.txt file.

Why sitemaps is a particular case

A sitemap is the strongest possible starting point, because it already enumerates the canonical public URLs. The work is pruning rather than discovery, which is a much easier problem.

A sitemap lists everything indexable, and llms.txt should list what is worth reading. Expect to cut most of it. A hundred-URL sitemap usually yields ten to twenty entries that genuinely help.

What to leave out of a sitemaps bundle

llms.txt is published. Everything in it is readable by anyone and by every AI system that fetches it, so the exclusions matter more than the inclusions.

• Every URL that exists only because a template generated it.
• Near-duplicate pages the sitemap treats as distinct.
• Deep pages that only make sense after reading a parent page you have not included.

Workflow

Add a sitemap when you need many related public pages, then filter down to the pages that are actually relevant.

• Collect the authoritative sitemaps pages or files.
• Remove private, duplicate, stale, or low-value material.
• Export a concise Markdown bundle.
• Use the bundle to draft or refresh llms.txt and llms-full.txt.

What to include

Prioritize stable, public, canonical information: product overview, important URLs, docs entrypoints, use cases, and contact details.

Avoid turning llms.txt into a full website dump. It should help AI systems discover the best sources, not replace your site.