MiniMax speaker Morgan Suo uses a financial research example to discuss evaluating a model beyond raw generation speedEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria.. She describes sparse attention, numerical precision, speculative decodingSpeculative decoding accelerates text generation by proposing several tokens cheaply and checking them together with the target model., concurrency and reasoning effortReasoning effort is the amount of internal computational work an AI model applies before producing an answer or action. as factors that affect the overall workflow.
The proposed evaluation checks sources, calculations, retrieval, retries and human correction as well as latency and costCost per completed task measures the total AI, tool, infrastructure, retry, and repair expense for each verified useful outcome.. Model-specific benchmark figures and performance claims are attributed to the speaker and her company.
Watch on YouTube
