What is a large language model text file?

Definition

A large language model text file gives an agent a compact starting point for understanding a website. It can identify the project, summarize its purpose, and link to the most useful documentation or machine-readable resources without requiring the agent to crawl and interpret every page.

The file should be hand-curated, current, and concise. It is not an access-control mechanism, a complete replacement for documentation, or a guarantee that every model will use it. Links should resolve to authoritative content, and changes should be tested from the perspective of several retrieval tools.

Acronyms and aliases

llms.txt acronym

Frequently asked questions

What should a large language model text file contain?

It should contain a concise project description and links to authoritative, current documentation or resources that an artificial intelligence system can retrieve.

Does an llms.txt file control artificial intelligence crawlers?

No. It supplies guidance and discovery links, while crawler access and indexing behavior are governed through other technical and provider-specific mechanisms.

Videos explaining large language model text file