Pierce Freeman and Richard Diehl Martinez discuss Ampersand, an AI assistant intended to help people coordinate. They caution that benchmark performanceA benchmark is a defined test or collection of tasks used to compare a system's performance under specified conditions. may not translate to real-world interactionsGeneralization is an AI system's ability to apply what it learned to new examples and conditions that were not part of its training experience. when systems are optimized for particular tests.
The hosts distinguish useful concrete information, such as a catering quote, from unsolicited mediation. They acknowledge that simple suggestions can be sensible while questioning the framing of an AI system as an orchestrator of people.
They characterize the example as a proof of concept and expect its prose to improve, but remain unconvinced about the role the assistant should play in group conversations.
Watch on YouTube
