Voice agents

Putting your agent on a call

Voice support in Compass Pro lets you serve any Parlant agent over a real call, from a browser or a phone line, without rebuilding it for speech. The same instructions, policies, and tools serve both text and voice. You bring the speech providers that turn audio into text and text back into speech, and LiveKit carries the audio. The agent can also speak first, and speak up on its own when a slow task finishes.

What you'll learn

• How a voice worker connects a LiveKit room to a Parlant session
• What you need before putting an agent on a call
• The three steps: a voice-mode agent, a worker, and a caller
• How the agent speaks first and speaks up on its own

Talk this page through with your coding agent

Media transport (the audio going back and forth) is handled by LiveKit. Parlant runs the conversation itself. On each turn, it takes what the caller said, generates a reply, and streams speakable text back to the caller, including proactive and opening speech, where the agent talks first.

You bring the speech providers. There are three: speech-to-text (STT), which turns the caller's audio into text; text-to-speech (TTS), which voices the reply; and voice-activity detection (VAD), which notices when the caller starts and stops speaking.

Your agent's instructions, policies, and tools are reused as they are across text and voice. The same agent serves both.

Note that you still need to test the voice experience on its own. Transcription errors, interruptions, pauses, and spoken wording can all change how a conversation feels, and what information actually reaches the agent.

How it works

Now that we know what voice adds, let's look at how a call actually reaches your agent.

Parlant does this with a voice worker: a process that bridges a LiveKit room to a Parlant session, one to one:

caller ⇄ LiveKit room ⇄ voice worker ⇄ Parlant session (agent)
             │                │
            STT  ──transcript──▶  Parlant turn
            TTS  ◀──speakable──   streamed reply chunks

A room is LiveKit's unit of a call, the place a caller's audio arrives. A session is Parlant's record of one conversation. The worker connects the two, and Parlant starts it from your own server process.

On each turn, your STT plugin transcribes the caller's audio, and Parlant feeds the transcript into the session as a turn. Parlant then generates the reply and streams speakable text back, and your TTS plugin voices it.

Note that the STT, TTS, and VAD objects are LiveKit plugins that you choose. Parlant passes them through untouched, so you are never locked into one vendor.

Prerequisites

Before you put an agent on a call, you need four things in place:

1. Compass Pro. Voice ships with it and depends on livekit-agents. Install the wheel you were given: pip install ./parlant_pro-<version>-py3-none-any.whl.
2. LiveKit plugins for your speech providers, for example pip install livekit-plugins-soniox livekit-plugins-silero.
3. A LiveKit server. Locally, livekit-server --dev needs no configuration. In production, set LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET for LiveKit Cloud or your own server.
4. API keys for your model provider and speech providers, in the environment.

Step by step

Once the prerequisites are in place, putting an agent on a call takes three steps: create a voice-mode agent, start a voice worker, and connect a caller. Your coding agent can do the first two from the examples below.

Create a voice-mode agent

The one required change to your agent is output_mode=p.OutputMode.VOICE.

Output mode is how the agent delivers a reply: whole, streamed as text, or, for voice, streamed as speakable segments (i.e., chunks the worker can hand to TTS as soon as they are ready). This is what lets the caller hear the start of a reply before the end is written:

import parlant.sdk as p

agent = await server.create_agent(
    id="car-sales-agent",
    name="Car Sales Agent",
    instructions="You help customers buy cars, asking questions to understand their needs.",
    output_mode=p.OutputMode.VOICE,   # replies stream as speakable segments
)

Start a voice worker

Next, hand the worker the server, the agent, and your speech plugins. Parlant starts it and keeps it running for the server's lifetime. There is nothing to await afterward.

The opening_instruction is a plain-language directive for the agent's first line, which it speaks as soon as the call connects:

import parlant.pro.voice.livekit as voice_livekit
from livekit.plugins import silero, soniox

worker = await voice_livekit.create_worker(
    server=server,
    agent=agent,
    stt=soniox.STT(),
    tts=soniox.TTS(voice="Ruby"),
    vad=silero.VAD.load(),
    opening_instruction=(
        "Warmly greet the customer, introduce yourself as the Car Sales Agent, "
        "and ask what kind of car they're looking for today."
    ),
)

Connect a caller

Finally, get a caller into a room. Each LiveKit room maps to one Parlant session: a caller joins a room, the worker joins it too, resolves the customer from the participant, and starts the conversation.

From a browser or app, join a room with a LiveKit client SDK. From a phone, connect a LiveKit SIP trunk so calls land in rooms.

By default, Parlant finds or creates the customer by participant identity, the name the caller joined the room with. If you would rather decide yourself, pass customer_resolver, a function of yours that maps a participant to a customer id.

You can also pass customer_id or session_id in the job metadata (i.e., the JSON your dispatch code attaches when it assigns a worker to a room). This binds the call to a known customer, or resumes an existing session.

Example: inventory lookup and a delayed search

To bring it all together, here is a car-sales agent that greets the caller, checks local inventory, and reports back after a delayed dealer search.

info

The tool results below are sample data. Replace them with your own inventory services; as written, they do not reserve a car or book a test drive.

Two things are new here.

First, the tools are ordinary Parlant tools, attached to policies (plain-language rules for what to do in a situation), exactly as they would be for a text agent.

Second, the slow tool returns at once, and later calls p.Agent.current.utter(...), which makes the agent speak on its own. Its rules argument carries the one-off instruction for what to say:

import asyncio

import parlant.sdk as p
import parlant.pro.voice.livekit as voice_livekit
from livekit.agents import AudioConfig, BuiltinAudioClip
from livekit.plugins import silero, soniox


@p.tool
async def check_availability(car_model: str) -> p.ToolResult:
    # Sample inventory: City is local; Touring needs a dealer search.
    if car_model.lower() == "city":
        return p.ToolResult(data={"model": car_model, "in_stock": True})
    return p.ToolResult(data={"model": car_model, "in_stock": False})


@p.tool
async def search_dealer_network(car_model: str) -> p.ToolResult:
    """Search the dealership network. Takes a while: tell the customer you'll update
    them when it's ready, and keep talking to them meanwhile."""

    async def run_search() -> None:
        await asyncio.sleep(5)
        if car_model.lower() == "touring":
            instruction = (
                "Tell the customer a Touring is listed at the North branch. "
                "Ask which location they prefer before offering further help."
            )
        else:
            instruction = f"Tell the customer no {car_model} was found in the sample dealer inventory."
        await p.Agent.current.utter(rules=[{"content": instruction}])

    asyncio.create_task(run_search())   # keep talking while the search runs
    return p.ToolResult(data={"status": "searching_dealer_network", "model": car_model})


async def main() -> None:
    async with p.Server(models={"*": "gemini"}, port=8810) as server:
        agent = await server.create_agent(
            id="car-sales-agent",
            name="Car Sales Agent",
            instructions="You help customers buy cars, understanding their needs first.",
            output_mode=p.OutputMode.VOICE,
        )

        await agent.create_policy(
            title="Test drives",
            content="If the customer asks about a model, check it with `check_availability` and explain the result.",
            tools=[check_availability],
        )
        await agent.create_policy(
            title="Dealer network",
            content="If the preferred model isn't available locally, search the dealership network.",
            tools=[search_dealer_network],
        )

        await voice_livekit.create_worker(
            server=server,
            agent=agent,
            opening_instruction=(
                "Warmly greet the customer, introduce yourself, and ask what car they want."
            ),
            stt=soniox.STT(),
            tts=soniox.TTS(voice="Ruby", speed=1.3),
            vad=silero.VAD.load(),
            ambient_sound=AudioConfig(BuiltinAudioClip.OFFICE_AMBIENCE, volume=1.0),   # looped through the call
            thinking_sound=AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING, volume=0.8),  # while the agent generates
        )


if __name__ == "__main__":
    asyncio.run(main())

Run it with a local LiveKit server and the relevant API keys, then join a room from a LiveKit client. Ask about City to exercise the local result, Touring for a successful delayed search, and any other model for the no-result response.

Proactive and opening speech

Now that we have seen a full example, let's look more closely at the two places where the agent speaks without being spoken to first. Voice is not purely reactive.

Opening speech

opening_instruction makes the agent speak first, right after the call connects. It is a directive the engine generates from, not a fixed string, so the greeting can be tailored per customer through variables (the per-customer values Parlant loads into the agent's context).

Learn more about variables

Guests and customers covers them.

Proactive speech

A slow tool can start its work, return immediately, and later call p.Agent.current.utter(...). Parlant then speaks the resulting message to the caller mid-conversation. That is how the dealer-network search above reports back. For a production implementation, keep the background task referenced and handle both search failures and reply-delivery failures. Asynchronous responses covers notification ordering, duplicate work, fixed failure messages, and recovery after a restart.

Note that the caller may talk over a proactive message, just as they might interrupt a person.

Learn more about the pattern

Asynchronous responses covers it in detail.

Configuration reference

Once you have a call working, here is everything you can tune on the worker. voice_livekit.create_worker(...) takes keyword arguments only:

Argument	Purpose
server, agent	The running server and a voice-mode agent. Required.
stt, tts	LiveKit speech-to-text and text-to-speech plugins. Required.
vad	A LiveKit voice-activity plugin. Recommended.
turn_detection	An optional end-of-turn model, passed through to LiveKit.
opening_instruction	A directive for the agent's first, proactive line.
ambient_sound, thinking_sound	Background audio looped through the call, and audio played while the agent generates.
warm_runners	Pre-warmed job runners. Default 0 for clean shutdown; raise for latency.
drain_timeout	Graceful-shutdown window in seconds.
connection	Explicit LiveKit credentials; otherwise environment variables, then the dev fallback.
customer_resolver	Maps a LiveKit participant to a Parlant customer id.
session_kwargs	Extra keyword arguments forwarded to LiveKit's AgentSession.

Configuring the plugins

Parlant hands the plugin objects to LiveKit untouched. This means languages, voices, endpointing (i.e., deciding when the caller has finished speaking), and speed are all configured on the plugin itself, as in soniox.TTS(voice="Ruby", speed=1.3).

Connecting to LiveKit

The worker resolves its connection in order: first an explicit LiveKitConnection(url=..., api_key=..., api_secret=..., agent_name=...), then the environment variables, then the local dev defaults.

Set agent_name for explicit dispatch (i.e., your dispatch code assigns the worker to rooms). Leave it empty and the worker auto-joins new rooms, which is the dev default.

Note that in production, you should set the environment variables and leave connection unset. This keeps credentials out of your code.

Production notes

• Graceful shutdown. Set drain_timeout to a number of seconds for rolling deploys. The worker then stops accepting new calls, and waits up to that long for active calls to finish. Left unset, it closes immediately and drops in-flight calls, which is right for local development, but not for production.
• Warm runners. Keep warm_runners at 0 unless latency matters more to you than a clean shutdown.
• Dispatch. For SIP or explicit dispatch, pass JSON job metadata with customer_id or session_id to bind a room to a specific customer, or to resume an existing session. Otherwise, Parlant resolves the customer from the participant identity.