Traditional retrieval-augmented generation treats context as a search problem. Documents are embedded into vectors, a user query retrieves semantically similar chunks, and those chunks are inserted into the model's prompt. This works well for narrow domains, but optimizing it increasingly resembles building a specialized search engine.
Large context windows do not remove the problem. Models still show context rot, lost-in-the-middle behavior, and weaker accuracy when relevant evidence is surrounded by similar distractors. Hardware and attention costs also make it impractical to load every potentially useful document into one prompt.
File-system agent harnesses reframe context delivery as navigation. A model receives tools to search, preview, open, and filter external files, allowing it to progressively discover evidence instead of depending on a separate retrieval layer to choose everything in advance.
Recursive language models extend the idea by placing a long prompt in a code-execution environment. The model can split material, call fresh sub-model instances over selected sections, aggregate their outputs, and verify the result. Łucek concludes that this improves accuracy on long analysis tasks, while conventional retrieval remains better when latency is the dominant constraint.
Watch on YouTube
