llms.txt: what it is, what it isn't, and how to add one in ten minutes

llms.txt is a short Markdown file that tells AI assistants what your site is and where the important pages are. What it does, what it does not do, and the version we run on flowsmith.online.

Published · 5 min read · Flowsmith editorialwebAI discoverabilitySEO

A dark library hall with a single folded map on a lectern at the entrance, lit by a small blue lamp.
Illustration generated for this article (fal.ai, Nano Banana 2): a map at the entrance, so nobody has to walk every aisle.
In short
  • /llms.txt is a Markdown file at the root of your site: a name, a one-paragraph summary and curated lists of links. It is written for language models and the agents that use them.
  • It does not control crawling (that is robots.txt), and it is not a ranking factor anyone has documented. Treat it as a cheap, accurate briefing, not as an SEO lever.
  • A good one takes ten minutes. The hard part is keeping it honest when the site changes.

When someone asks an AI assistant "what does Flowsmith do?", the assistant either finds a clear answer or pieces one together from navigation menus, cookie banners and old pages. llms.txt is a small, deliberate attempt to give it the clear answer. We run one on flowsmith.online and generate one for every site we launch. This article explains what the file is for, what it is not for, and how to write one that helps.

What llms.txt is

The convention was proposed by Jeremy Howard of Answer.AI in September 2024 and is documented at llmstxt.org. The idea is simple. Websites are built for people and browsers. Language models work better with short, clean Markdown. So you put a curated Markdown overview at a predictable address, /llms.txt, the same way robots.txt and sitemap.xml live at predictable addresses.

The format is plain Markdown in a fixed order:

  1. An H1 with the name of the site or project (the only required part).
  2. A blockquote with a short summary: what this is, in one or two sentences.
  3. Optional paragraphs with details a model should know.
  4. H2 sections containing lists of links, each with an optional one-line note.
  5. An optional section titled ## Optional for links that can be skipped when context is short.

A common companion is /llms-full.txt, which contains the actual content of the key pages in one file, so an agent can read everything in a single request.

What it is not

Most of the confusion around llms.txt comes from expecting it to do something it does not do.

  • It is not robots.txt. It grants and blocks nothing. Whether GPTBot, ClaudeBot, PerplexityBot or Google-Extended may fetch your pages is decided in robots.txt. Ours allows them by name, because we want assistants to know what Flowsmith is.
  • It is not a sitemap. The sitemap lists every indexable URL for crawlers. llms.txt lists the few pages that explain you, with context. Ours lists the lessons, manuals and articles that explain us, each with one line of context; the sitemap lists every URL in both languages with no context at all.
  • It is not a proven ranking factor. We are not aware of any major search or AI provider documenting that it uses llms.txt for ranking or citation. Anyone selling it as a traffic switch is overselling it.

Why bother, then? Because it costs almost nothing, and the readers who do fetch it directly benefit a lot: coding agents, research tools, assistants that browse when asked about you, and developers who paste your docs into a chat. A clean briefing means fewer wrong descriptions of what you do.

The version we run

Here is a shortened version of the live file on flowsmith.online:

# Flowsmith

> Flowsmith is KH Group's automation studio. We design and operate AI-assisted
> workflows — inbox handling, drafting, operations — that fit how a team already
> works, with a human in the loop for anything that leaves the building.

Flowsmith is the automation brand of KH Group (KH s.r.o., Praha 1, Czech Republic). …

## Site
- [Flowsmith — home](https://flowsmith.online/): what we automate, how an engagement works, and the contact form.
- [Contact](https://flowsmith.online/#contact): describe the busywork; we reply with a proposal.

## What we do
- AI in the loop — language models draft and classify; people approve before anything is sent.

## Academy
- [Working with an AI agent](https://flowsmith.online/academy/working-with-ai-agents/): Lesson 1: the simple loop …

## Blog
- [Human-in-the-loop automation](https://flowsmith.online/blog/human-in-the-loop-automation/): …

## Czech (čeština)
English is canonical at unprefixed URLs; Czech lives under /cs/ with the same path.

## Optional
- [Sitemap](https://flowsmith.online/sitemap.xml): every page with its hreflang counterpart.

Three choices in it are deliberate:

  • The summary says what we do and how, including the part that sets us apart (a person approves anything outbound). That is the sentence we most want an assistant to repeat accurately.
  • A Czech section. The site is bilingual, so we tell models where the Czech versions live and that English is canonical, instead of letting them guess. We keep the same rules in the page markup with hreflang links.
  • A one-line description per link, taken from the page's own meta description, so the briefing and the page never disagree about what the page is.

How to write yours in ten minutes

  1. Write the blockquote first. One or two sentences: who you are, what you do, for whom. If you cannot write it, the problem is not the file.
  2. Pick five to fifteen links that explain you: services, pricing or how to engage, documentation, contact. Skip legal pages, tag archives and anything a person would not read to understand you.
  3. Add one plain line per link saying what is on the page. Write it for a reader with no context.
  4. Use absolute URLs and check that every link returns a page, not a redirect chain or a 404.
  5. Put it at the root, served as plain text, and link it from nowhere in particular. Agents know where to look.
  6. Make sure robots.txt agrees. A perfect llms.txt behind a robots.txt that blocks AI crawlers will not help.

You do not need to write any of this by hand. Ask your AI agent to "set up SEO and AI discoverability for this site: robots.txt with an AI-crawler allowlist, llms.txt, sitemap.xml, Open Graph, canonical and hreflang". Our Publishing & discoverability manual explains each file and the one-pass recipe we use.

Mistakes we see, and made

  • Marketing copy instead of facts. "Revolutionary platform for tomorrow's leaders" tells a model nothing. Say what you do in plain words.
  • Letting it rot. Our sitemap is generated from the HTML files on every deploy, but llms.txt is curated by hand, so it changes only when someone changes it. We treat it like a README: whenever a section of the site is added or renamed, the same commit updates llms.txt. This blog was added to it in the commit that launched the blog.
  • Listing everything. A 400-link llms.txt is a worse sitemap. Curate it.
  • Forgetting the other languages. If you have translated pages, say where they are. Otherwise an assistant may describe your English site to a Czech user, or the other way round.

Where it fits

llms.txt is one of the finishing files that make a site findable and shareable, alongside a favicon, Open Graph tags, a sitemap and structured data. It works best on a site that is already fast and clearly structured. If you are starting from nothing, the Academy lesson From zero to a live website takes you from an empty folder to a published site, and Make your project AI-friendly covers the same idea for your code: a few plain text files that brief an agent before it starts work.

Want this running in your company?

Flowsmith designs and operates automations like this for small and mid-sized teams — with a person approving anything that leaves the building.

More from the blog