Sohail Shaikh and Ankush Rastogi discuss the cost and reliability problems that arise when an agent receives a large catalog of tool definitions on every request.
They propose indexing tool descriptions, retrieving a relevant subset for each task, and injecting schemas just in time. They present examples and benchmark results, and discuss measuring selection accuracy, latency and token costs.
The speakers also discuss maintaining descriptions, monitoring retrieval misses and using fallback searches. They say smaller toolsets may not need a router.
Watch on YouTube
