Instrument voice agents

A voice call is an Agent Analytics session. Each caller utterance is an [Agent] User Message and each agent reply is an [Agent] AI Response, with the transcript as the message content. Voice turn properties add what a transcript can't show: how long the caller spoke, how fast the agent started talking, and whether either side cut the other off. Built-in signals, evaluators, and charts then work on calls the same way they work on chat.

Use the call ID as the session ID, so every turn of one call lands in one session.

This page is for voice agents you run yourself. If your agent runs on Vapi, Retell AI, ElevenLabs Agents, or Bland AI, a scheduled job imports finished calls with the same voice properties instead. Refer to Voice agent platforms.

Set up with your AI coding agent (recommended)

Paste this prompt into your AI coding agent, such as Cursor, Claude Code, Windsurf, GitHub Copilot, or Codex. The agent installs the SDK, reads the bundled instructions, finds every turn in your voice loop, and instruments it. It also writes a verification test.

Instrument this voice agent with Amplitude Agent Analytics using the Node SDK.

Fetch and follow https://raw.githubusercontent.com/amplitude/Amplitude-AI-Node/main/amplitude-ai.md
Follow the step "Voice agents" for the voice turns, then install the SDK as that guide says:

npm install @amplitude/ai @amplitude/analytics-node

Review the changes it proposes, then verify your instrumentation.

Record voice turns

Pass the helper's output as the turn's event properties. Leave a field out when you don't know it. An absent field means unknown, not false.

from amplitude_ai import voice_turn_properties
from amplitude_ai.core.tracking import track_ai_message

props = voice_turn_properties(
    direction="inbound",
    time_to_first_audio_ms=280,
    interrupted=True,
    interrupted_by="user",
    end_of_turn_reason="interrupted",
)
track_ai_message(
    amplitude=amp,
    user_id="caller-42",
    model_name="gpt-realtime",
    provider="openai",
    response_content=reply_text,
    session_id=call_id,
    event_properties=props,
)

The helper rejects values outside the contract, such as an unknown end-of-turn reason or a transcript confidence above 1. For every property and its allowed values, refer to Voice turn properties.

Add stage latency and call timing

Time to first audio is what the caller hears. To find which stage of your pipeline caused a slow reply, and where each turn sits in the call, pass these optional fields too. All values are milliseconds and must be 0 or greater.

The helper rejects an end offset that's before the start offset.

props = voice_turn_properties(
    direction="inbound",
    time_to_first_audio_ms=910,
    stt_ms=180,
    llm_ttft_ms=420,
    tts_ttfb_ms=160,
    turn_start_offset_ms=12_400,
    turn_end_offset_ms=15_900,
    pre_reply_silence_ms=910,
    end_of_turn_reason="completed",
)

Measure both offsets from the same call start for every turn, so the session view can draw the call timeline.

Map speech-to-speech model events

If your agent streams audio to a speech-to-speech model directly, a mapper turns the model's server events into finished turns. Feed it every event you already receive. It returns zero or more turns, each with a role, the transcript text, and voice turn properties. The mappers send nothing themselves and don't depend on the provider's SDK.

OpenAI Realtime

RealtimeTurnMapper handles both the GA and beta event names. It sets speech duration from voice activity detection, time to first audio from the first audio chunk, and marks a reply the caller cut off as interrupted by the user. A failed or incomplete response ends with the reason error.

from amplitude_ai.integrations.openai_realtime import RealtimeTurnMapper

mapper = RealtimeTurnMapper(direction="in_app")
async for event in connection:
    for turn in mapper.feed(event.model_dump()):
        if turn.role == "user":
            session.track_user_message(turn.text, event_properties=turn.event_properties)
        else:
            session.track_ai_message(
                turn.text,
                model=turn.model or "gpt-realtime",
                provider="openai",
                latency_ms=0,
                event_properties=turn.event_properties,
            )

Amazon Nova Sonic

NovaSonicTurnMapper reads the output events of a Bedrock bidirectional stream, with or without the outer event wrapper. Nova Sonic sends the agent's text twice: a speculative preview before the audio and a final transcript of what it spoke. The mapper keeps only the final transcript. It measures time to first audio from the end of the caller's transcript, and marks a reply the caller cut off as interrupted by the user.

import json

from amplitude_ai.integrations.nova_sonic import NovaSonicTurnMapper

mapper = NovaSonicTurnMapper(direction="inbound")
async for chunk in stream:
    event = json.loads(chunk.value.bytes_.decode())
    for turn in mapper.feed(event):
        if turn.role == "user":
            session.track_user_message(turn.text, event_properties=turn.event_properties)
        else:
            session.track_ai_message(
                turn.text,
                model="amazon.nova-sonic-v1:0",
                provider="bedrock",
                latency_ms=0,
                event_properties=turn.event_properties,
            )

Track voice frameworks

LiveKit Agents and Pipecat already report turn timing. The SDK's LiveKit and Pipecat integrations read it and track each caller and agent turn with voice turn properties, including stage latency and call timeline offsets. Neither integration adds the framework as an SDK dependency.

LiveKit Agents

attach subscribes to conversation_item_added on a LiveKit AgentSession and tracks every finished caller and agent message. It reads speaking start and stop times, e2e_latency for time to first audio, the caller's transcription delay, the model's time to first token, and the speech synthesizer's time to first byte from each message's metrics. A message the caller cut off ends as interrupted by the user.

from amplitude_ai.integrations.livekit import attach

agent = ai.agent("support-voice", user_id=customer_id, session_id=ctx.room.name)
attach(lk_session, agent, model="gpt-4o-mini", provider="openai")

Pass the model and provider your agent's LLM uses, so cost and model charts work. direction defaults to inbound; pass direction="outbound" or direction="in_app" for other calls.

Pipecat

PipecatVoiceTracker reads the frames a Pipecat pipeline already pushes and tracks one caller turn and one agent turn per exchange. Add it to the pipeline task with amplitude_observer:

from amplitude_ai.integrations.pipecat import PipecatVoiceTracker, amplitude_observer

agent = ai.agent("support-voice", user_id=customer_id, session_id=call_id)
tracker = PipecatVoiceTracker(agent, direction="inbound", model="gpt-4o", provider="openai")
task = PipelineTask(pipeline, observers=[amplitude_observer(tracker)])

The tracker measures offsets from the first frame it sees. An observer sees each frame once per hop, and the tracker ignores a frame it already saw.

Send LiveKit or Pipecat traces over OpenTelemetry

LiveKit Agents and Pipecat can also export OpenTelemetry traces. If you already export them, add Amplitude as an OTLP exporter instead of using the SDK integration. Refer to Send OpenTelemetry traces to Agent Analytics for the endpoint and headers.

• Voice timing from the framework's spans attaches to the [Agent] User Message and [Agent] AI Response of each turn automatically.
• Use a batch span processor, so a turn and its child spans export together.
• Send each call's turns through one path only: the SDK integration or OTLP. If you use both, every turn counts twice.

Attach the call recording

Attach the recording URL to the last turn of the call, whichever side spoke last. A call that ends on a hang-up or a transfer can end on a caller turn.

from amplitude_ai import recording_attachment

session.track_ai_message(
    closing_text,
    model="gpt-realtime",
    provider="openai",
    latency_ms=0,
    attachments=[recording_attachment(recording_url, mime_type="audio/wav")],
)

The recording is a reference, like a Session Replay asset that isn't inlined. Amplitude stores the URL as you pass it and never fetches it, so the link plays only for a viewer who can reach the host. Like other attachment content, the URL ships only in full privacy mode.

A signed link from a voice platform expires, and until then anyone holding it can play the call. For a durable reference, pass a URL your own access controls protect, such as your CDN or a host reachable only on your VPN.

Record call outcomes

The built-in Task Completed signal judges each call from its transcript, with no setup. If your voice platform also reports its own verdict on the call, send it as a session score so you can compare the two:

ai.score(
    user_id="caller-42",
    session_id=call_id,
    name="platform-call-success",
    value=1 if call_succeeded else 0,
    target_id=call_id,
    target_type="session",
    source="ai",
)

For how scores appear in Agent Analytics, refer to Collect user feedback.

Analyze voice calls

After a voice session closes, enrichment rolls the turn properties up onto its [Agent] Session Record: average and slowest time to first audio, average and 95th percentile stage latency, the slowest stage, total pre-reply silence, overlap count, call duration, interruption and yield counts, total speech time, and the final end-of-turn reason. It copies the same rollups onto each [Agent] Evaluator Result, so you can segment evaluator outcomes by them too. Chat sessions don't carry the rollups. For the full list, refer to Session Record.

Start with these charts on [Agent] Session Record:

• Voice and chat task completion: The share of sessions where [Agent] Task Completed is true, grouped by [Agent] Channel. Chat sessions show (none) for the channel.
• Latency by direction: The average of [Agent] Voice Avg Time To First Audio Ms, grouped by [Agent] Voice Direction. Compare it with [Agent] Voice Max Time To First Audio Ms to find calls with a single slow turn.
• Barge-in rate: The share of voice sessions where [Agent] Voice User Interruption Count is greater than 0, grouped by [Agent] Root Agent Name.
• How calls end: Sessions grouped by [Agent] Voice Final End Of Turn Reason. A rise in silence or error points to dropped calls.
• Where latency comes from: Sessions grouped by [Agent] Voice Slowest Stage, or the average of [Agent] Voice P95 LLM TTFT Ms over time.

Voice issue flags

Enrichment also sets these flags on the Session Record of a voice call:

The Voice issues view in Agent Analytics lists the calls these flags catch.

Review a call

The session view of a voice session shows a call timeline of caller and agent turns, built from the turn offsets. When a turn carries a call recording, the view also shows a recording player. The recording plays in your browser from your URL; Amplitude never fetches or stores the audio. The player appears only for sessions sent in full privacy mode, the only mode that ships the recording URL.