The AI Automators compares a small locally fine-tuned Laya decision model with cloud-hosted Jev on choosing the next action from 30 customer-support tools. The experiment separates training, validation, calibration and test data rather than judging the model on conversations used to train it.
With basic state, the presenter reports 82.2 percent accuracy for fine-tuned Laya and 67.3 percent for Jev on 1,000 test chats. Richer context changes the result: Jev reaches 87.4 percent and Laya 86.1 percent. These are results from one dataset and configuration, not evidence that either model wins across all support systems.
The second part examines whether model confidence is trustworthy enough to permit automatic action. Temperature calibration reduces confident errors at a 90 percent gate but increases human escalations, demonstrating a practical tradeoff between mistakes and staff workload. Course promotions and community appeals are omitted.
Watch on YouTube
