Jared Joselowitz describes Ufonia's approach to evaluating a clinical conversational agentEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. before exposing patients to changes. The title's million-patient figure refers to a reported contract for future scale. The engineering argument begins with defining possible harm, such as missed warning signs or invented advice, because an already completed patient conversation cannot simply be undone by rolling back software.
Jared Joselowitz presents Matrix, a simulation framework in which an LLM plays a patient under a defined clinical scenarioSynthetic data is information generated to represent examples or situations rather than collected directly from the events it describes. and converses with the target agent. The process checks whether the simulated patient follows instructions and uses patient and public feedback to assess realism. The talk emphasizes that realistic testing needs varied personas and communication styles, including rare situations that ordinary call data may not capture.
Jared Joselowitz describes an automated judge that returns structured hazard decisions and reasons, then explains how the team compared it with clinician labelsHuman judgment in AI is the accountable interpretation and decision-making people contribute when setting goals, evaluating evidence, and managing consequences.. The reported performance comes from the team's study. Evaluation priorities are tied to the consequences of an error, so missing a dangerous condition can receive a different penalty from unnecessarily asking additional questions.
Jared Joselowitz connects those evaluation results to automated prompt optimizationPrompt engineering is the practice of designing and refining instructions, context, examples, and constraints to obtain useful AI outputs.. A model reviews failures, proposes prompt changes and keeps useful candidatesAn AI feedback loop occurs when AI outputs or their consequences become inputs that influence later model or agent behavior. against an explicit metric, reducing reliance on manual adjustments. The proposed workflow retains prompt versions, data and decision traces so improvements can be inspected and connected to the hazards they are intended to address.
Jared Joselowitz explicitly states that successful simulation does not establish real patient benefit. The described progression moves through user testing, supervised clinical evaluation and monitored deployment, with autonomy increasing according to evidence. New languages and voice behaviors can introduce fresh hazards, including interruptions that prevent important information from being completed, so the safety case needs continuing revision.
Watch on YouTube
