Marina Petzel argues that latency, errors, traffic and saturation remain useful but do not show whether a generative AI response is accurate, safe or affordable. She describes output variability and changing token costs as additional challenges.
She discusses token growth, model choiceModel selection chooses the AI model whose capability, quality, cost, speed, safety, and operating constraints best fit a task. and repeated calls as potential cost drivers, and proposes attributing spending to features, users, models and environments. For safety she describes monitoring prompt injectionPrompt injection occurs when untrusted content attempts to redirect an AI system's behaviour beyond the authority of that content., sensitive information, toxic content and jailbreak attempts.
For quality, Petzel discusses hallucinationA hallucination is an AI output that presents false, unsupported, or invented information as though it were reliable. checks, relevance, user feedback, answer completeness and retrieval qualityRetrieval is the process of selecting relevant stored information and returning it to an AI system for the current task.. She presents manual and automated evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. as complementary and ends with a Datadog product mention.
Watch on YouTube
