A large language model text file gives an agent a compact starting point for understanding a website. It can identify the project, summarize its purpose, and link to the most useful documentation or machine-readable resources without requiring the agent to crawl and interpret every page.
The file should be hand-curated, current, and concise. It is not an access-control mechanism, a complete replacement for documentation, or a guarantee that every model will use it. Links should resolve to authoritative content, and changes should be tested from the perspective of several retrieval tools.
ELI5
A large language model text file is a short guide that points AI systems to the most useful information about a website. It usually identifies the site and links to authoritative pages so an AI tool has a sensible place to begin.
For example, a software project can publish an llms.txt file that links to its official documentation, API guide and support policy. The file does not grant access or replace those pages, so its links and descriptions still need regular maintenance.
