What is capability-to-cost ratio?

Definition

Capability-to-cost ratio relates measured ability or useful outcomes to spending. Cost can include model usage, tools, infrastructure, retries, supervision, and repair, while capability should be measured on tasks representative of the intended workload.

A stronger model is better value only when its added performance repays its added cost for the actual job. The ratio can change across coding, research, computer use, routine automation, and different effort settings, so one global ranking is rarely sufficient.

ELI5

A capability-to-cost ratio compares how much useful AI performance a model provides with what it costs to use. It considers whether tasks are completed reliably, not just the price of one token or a headline benchmark score.

For example, a cheaper model may have a better ratio for routine document edits if it succeeds without much correction. A more expensive model may be better for difficult architecture work when the smaller model's errors create more review and rework.

What belongs in the cost side of this ratio?

Include model usage, tools, infrastructure, retries, failures, supervision, review, repair, and other spending needed for a verified result.

Why does capability-to-cost ratio vary by workload?

Models have different strengths, and each workload values different capabilities, quality thresholds, latency, tools, reliability, and failure costs.

  1. Pat Simmons beside the headline Fable 5.1 Cheaper Not Better?
    How Fable 5.1 Compares in Hands-On Tests
    Pat Simmons29m 59s2 VIEWS
  2. Nathaniel Whittemore beside the headline Fable 5.1 Earns Its Place
  3. A flat lightning bolt illustration beside the words Fast AI Gets Capable
Definition card for capability-to-cost ratio: What is capability-to-cost ratio?

Capability-to-cost ratio compares the useful performance an AI system delivers with the total cost required to obtain it.