Eugene Cheah presents an inference provider's perspective on AI operating costs and discloses that Featherless sells inference. He begins with attribution: separate usage by department, team and task before changing models or capacity. His suggested spending threshold for prioritizing optimization is a rule of thumb, not a universal budget requirement.
Eugene Cheah distinguishes long-context agent workloads from high-volume process automation. He describes splitting planning, implementation and review across appropriately capable models, using task-specific evaluations to choose smaller models, and testing fine-tuning or classifiers where a workload warrants them. Vendor anecdotes about savings and quality remain reported examples rather than independently reproduced benchmarks.
Eugene Cheah explains the tradeoff between per-request generation speed, concurrent throughputAI inference throughput measures how much model-serving work a system completes per unit of time. and infrastructure cost. Dedicated capacity can suit a measurable base load, while variable demand may need a different arrangement. He stresses that traffic shape, caching, latency requirements and deployment constraints determine whether a proposed configuration is appropriate.
Eugene Cheah warns that rapidly producing code, slides or reports can simply move work onto reviewers. He advocates manageable changes, testing and responsibility for generated output. Token spending is not equivalent to completed workCost per completed task measures the total AI, tool, infrastructure, retry, and repair expense for each verified useful outcome., and measuring organizational outcomes remains difficult even after usage becomes visible.
Eugene Cheah answers questions about supervised fine-tuning, golden examples, agent harness flexibility, dedicated caches and task-based routingAI model routing sends each request to a model chosen for that request's complexity, cost, speed, privacy, or other requirements.. He recommends evaluating results rather than assuming that a particular tuning method is best, and notes that switching models too frequently can undermine cache efficiency. Brief audience questions provide context without introducing additional principal presenters.
Watch on YouTube
