---
url: https://findutils.com/guides/llms-txt-generator
title: "How to Create an llms.txt File for Your Website"
description: "Write a valid llms.txt in the llmstxt.org format: an H1, a summary and sections of described links. What the file does, what it cannot promise, and where it goes."
category: seo
content_type: guide
guide_type: subtopic
cluster: developer
locale: en
read_time: 7
status: published
author: "olgunozoktas"
published_at: 2026-09-26T12:00:00Z
excerpt: "llms.txt is a short Markdown file at the root of a site that tells language models and agents what the site is and which pages are worth reading. Writing one takes minutes; getting the format right, keeping it small and being realistic about what it does takes a little more care."
tag_ids: ["llms-txt", "geo", "seo", "ai-crawlers"]
tags: ["llms.txt", "GEO", "SEO", "AI Crawlers"]
primary_keyword: "create llms.txt file"
secondary_keywords: ["llms.txt generator", "llms.txt format", "llms.txt example", "what is llms.txt", "llms.txt vs robots.txt"]
tool_tag: "llms-txt-generator"
related_tool: "llms-txt-generator"
related_tools: ["llms-txt-generator", "geo-analyzer", "robots-txt-generator", "robots-txt-tester", "bot-crawl-checker"]
og_image: "/images/content/guides/llms-txt-generator-cover-20260926.webp"
image_alt: "A small lantern robot reading a blank scroll on a pedestal beside a miniature house, with violet light linking the scroll to floating blank cards."
updated_at: "2026-09-26T12:00:00Z"
---
To create an llms.txt file, write a Markdown file with your site's name as the only H1, a one- or two-sentence summary in a blockquote, and H2 sections that list your key pages as `- [title](url): description` lines, then serve it at `https://yoursite.com/llms.txt`. FindUtils [llms.txt Generator](/seo/llms-txt-generator/) builds that file from a form, resolves your links against your domain, and warns about duplicates, off-site links and placeholder titles. It also gives you a matching robots.txt snippet. The generator fetches nothing and runs in your browser.
This guide covers what the format is, what it can and cannot do for you, how to fill in the generator, and where to put the result.
## What Is llms.txt?
llms.txt is a proposal published by Jeremy Howard at [llmstxt.org](https://llmstxt.org/) in September 2024. It describes a Markdown file at the root path `/llms.txt` of a website (or at a subpath such as `/docs/llms.txt`) that gives language models a short, curated index of the site. The format, in order:
1. **An H1 with the name of the site or project.** The proposal calls this the only required section.
2. **A blockquote** with a short summary.
3. **Optional prose**: paragraphs or lists, but no headings.
4. **H2 sections of "file lists"**: Markdown links, each optionally followed by `:` and a note.
5. **An `## Optional` section**, by convention holding secondary links that a tool can skip when its context is short.
The proposal also suggests offering clean Markdown versions of pages, for example at the page URL with `.md` appended.
## What llms.txt Does and Does Not Do
llms.txt is a proposal, not a standard from a standards body, and no search engine or AI provider guarantees that its systems read the file or cite sites that publish one. What it offers is a clean, human-written index for any agent or tool that does look for it.
| File | Tells machines | Enforced? |
|---|---|---|
| `robots.txt` | Which paths a crawler may fetch | Advisory; crawlers choose to obey |
| `sitemap.xml` | Every URL you want discovered, with dates | A hint for search engines |
| `llms.txt` | What the site is and which pages matter, in prose | Nothing reads it by obligation |
That makes llms.txt cheap to add and easy to overrate. Publish it because a clear index of your best pages is useful, not because it promises citations.
## How to Build the File
### Step 1: Name the site and the origin
Open the [llms.txt Generator](/seo/llms-txt-generator/). Enter the site name, which becomes the H1, and the origin, such as `https://docs.example.com`. Links you write as `/guides/start` are resolved against that origin.
### Step 2: Write the summary
Write one or two sentences for the blockquote. Say what the site is and, if the name is ambiguous, what it is not. The generator warns above 320 characters.
### Step 3: Add sections and links
Add H2 sections such as Guides, API or Pricing, and give each link a title, a URL and one sentence saying why an agent should open it. Put secondary links, such as a changelog, in the Optional section; it is always written last.
### Step 4: Fix the warnings and copy the outputs
Read the warnings, then copy or download `llms.txt`, and copy the robots.txt snippet and the header snippet if you want them.
## A Worked Example
Input: site name `Example Docs`, origin `https://docs.example.com`, summary `Example Docs is the public documentation for Example.`, a Guides section with `Start` at `/start` ("Begin here."), an optional link `Changelog` at `/changelog` ("Release history."), and the sitemap URL `https://docs.example.com/sitemap.xml`.
The generated `llms.txt`:
```markdown
# Example Docs
> Example Docs is the public documentation for Example.
## Guides
- [Start](https://docs.example.com/start): Begin here.
## Optional
- [Changelog](https://docs.example.com/changelog): Release history.
```
The robots.txt snippet:
```text
User-agent: GPTBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: Applebot-Extended
Allow: /
Sitemap: https://docs.example.com/sitemap.xml
```
The header snippet is `Link: ; rel="describedby"; type="text/plain"`, and the HTML version is ``. Both are pointers for tools that look for one; nothing requires an agent to follow them.
## What the Generator Checks
| Problem | Result |
|---|---|
| Origin missing, not absolute, or not http(s) | Error |
| A link URL that is relative without a leading `/` (`page.html`) | Error |
| A URL with an angle-bracket placeholder (`https://x.com/`) | Error |
| A link with no title | Error |
| A link on another host than the origin | Warning |
| The same URL listed twice | Warning |
| A title copied from a template, such as `Example Corp` or `Page Title` | Warning |
| A link with no description | Warning |
| A summary over 320 characters, or a file over about 100 KB | Warning |
| Headings inside the prose | Escaped to plain text, with a warning, so they cannot start a new section |
| An empty section | Left out, with a warning |
`[`, `]` and `\` in titles are escaped, and parentheses in URLs are percent-encoded, so a link cannot break the Markdown. The generator checks the shape of each URL only: it does not fetch the pages, so it cannot tell you a link returns 404.
## Decide Which AI Crawlers to Allow
The robots.txt snippet allows five AI user-agent tokens by default. They do different jobs, according to their operators:
- **GPTBot** (OpenAI) crawls content that [may be used in training](https://developers.openai.com/api/docs/bots) OpenAI's foundation models. ChatGPT search uses a separate token, OAI-SearchBot.
- **ClaudeBot** (Anthropic) collects content that [could contribute to model training](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler).
- **PerplexityBot** [surfaces and links websites](https://docs.perplexity.ai/guides/bots) in Perplexity's search results.
- **Google-Extended** controls whether content may be used for Gemini training and grounding, and Google states it [does not affect Google Search](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers).
- **Applebot-Extended** does not crawl pages; Apple uses it to let publishers [opt out of training](https://support.apple.com/en-us/119829) Apple's foundation models on content Applebot already collected.
Allowing a training crawler is a choice about your content, not a technical requirement for llms.txt. Delete any group you do not want before pasting the snippet into your robots.txt, and check the result with [robots.txt Tester](/seo/robots-txt-tester/).
The optional Content-Signal line (for example `search=yes, ai-input=yes, ai-train=no`) states your preferences in robots.txt. It is a newer proposal, and nothing obliges a crawler to honour it.
## Where to Put the File
Serve the file at the root of the origin, `https://yoursite.com/llms.txt`, as plain text or Markdown, with a 200 status and no login. In most frameworks that means placing it in the public or static folder. Keep it short: the useful file lists the pages you would want quoted, not every URL on the site. That is what a sitemap, built with the [XML Sitemap Generator](/seo/xml-sitemap-generator/), is for.
## Common Mistakes
**Listing everything.** An llms.txt with hundreds of links is a sitemap with extra steps. Pick the pages that explain the site.
**Leaving template text.** `Example Corp`, `Page Title` and `` placeholders ship more often than you would think. The generator flags them.
**Links without descriptions.** The one sentence after the link is what tells a reader why to open it.
**Expecting traffic.** Publishing llms.txt does not make any AI system cite you. Measure what you can, such as whether AI crawlers can reach your pages, with the [GEO Analyzer](/seo/geo-analyzer/).
## Use It From Code
The generator is the `llms_txt_generate` tool on the FindUtils REST API and MCP server.
```bash
curl -X POST https://api.findutils.com/api/tools/llms-txt-generate/execute \
-H "Content-Type: application/json" \
-d '{"site_name":"Example Docs","origin":"https://docs.example.com","summary":"Example Docs is the public documentation for Example.","sections":[{"heading":"Guides","links":[{"title":"Start","url":"/start","description":"Begin here."}]}]}'
```
Other arguments: `details`, `optional_links`, `content_signal` and `sitemap_url`. The response holds `llms_txt`, `warnings`, `robots_snippet`, `link_header` and `html_link`. See the [API reference](/api/llms-txt-generate/) and the [MCP reference](/mcp/llms-txt-generate/).
## Tools Used in This Guide
| Tool | Use |
|---|---|
| [llms.txt Generator](/seo/llms-txt-generator/) | Build a valid llms.txt and matching snippets |
| [GEO Analyzer](/seo/geo-analyzer/) | Check a page's readiness for AI search |
| [Robots.txt Generator](/seo/robots-txt-generator/) | Build the robots.txt the snippet goes into |
| [robots.txt Tester](/seo/robots-txt-tester/) | Check which crawlers your robots.txt allows |
| [Bot Crawl Checker](/network/bot-crawl-checker/) | Check whether a live URL is reachable for well-known search, AI and social bots |
## FAQ
### Is llms.txt required for AI search?
No. It is an optional proposal. No search engine or AI provider guarantees it reads the file, and pages are found and cited without one.
### Where does llms.txt go?
At the root of your site, `https://yoursite.com/llms.txt`. The proposal also allows a subpath, such as `/docs/llms.txt`, for a section of a site.
### What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers which paths they may fetch. llms.txt tells a reader what the site is and which pages matter. One is about access, the other about orientation.
### Does the generator check that my links work?
No. It checks that each URL is well formed and on your origin, but fetches nothing, so a broken link is not detected.
### Should I allow GPTBot and ClaudeBot?
That is your decision. Both operators describe these crawlers as collecting content that may be used for model training. Remove their groups from the snippet if you do not want that.
## Next Steps
Build your file with the [llms.txt Generator](/seo/llms-txt-generator/), then read [How to Make Your Website AI-Agent Ready](/guides/how-to-make-your-website-ai-agent-ready/) for the other files that sit beside it.