Rishabh Bhargava on Engineering Voice Agents

AI Engineer24m 35s
Rishabh Bhargava beside the blue and white headline Voice Agent Engineering on black.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Together AI voice team lead Rishabh Bhargava describes a cascading architecture that streams speech through transcription, a language model and speech synthesis. He discusses turn detectionTurn-taking coordinates when each participant speaks in a conversation., latency and accurate recognition of important words.

    Bhargava examines time to first token and audio, network proximity and scaling long-lived conversations. He presents latency figures as examples rather than universal production guarantees.

    He contrasts the pipeline with speech-to-speech models, discussing potential gains in conversational nuance alongside instruction-following, tool-calling and evaluation challengesEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria.. Audience questions cover guardrails, model placement and observability.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows100
    • Coding & tools44.7
    • Models & learning94.7
    • Infrastructure & compute68.1
    • Research & evaluation55.3
    • Safety & security25.5
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    Rishabh Bhargava beside the blue and white headline Voice Agent Engineering on black. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 31 May 2026 and duration 24m 35s.

    Rishabh Bhargava explains how voice-agent pipelines balance latency, quality and reliability, and compares them with speech-to-speech models.