Dianhuan Lin distinguishes meaningful disagreement about a decision from harmless variation in wording. A label may be ambiguous because business policy, relevant facts or the decision boundary is unclear. Repeated predictions can expose that uncertainty rather than simply proving that a model is unreliable.
Dianhuan Lin adapts active learning to prioritize human review where model verdicts differ. Self-reported confidence is not a reliable substitute for identifying what the model does not know. Expert clarification can supply missing facts, labels and organization-specific rules.
Dianhuan Lin describes semantic memory for distilled policies and episodic memory for previous reviewed cases. These approaches complement one another and can improve consistency without relying only on expensive fine-tuning. Her small alert-review example illustrates the approach but does not guarantee elimination of inconsistent decisions.
Watch on YouTube
