Voice Agent

Example of using LiveKit to enable bidirectional voice chat with an AI agent built with LiveKit Agents.

The agent in this example can interact with hardware in response to user requests. Below is an example of a conversation between a user and the agent:

User: What is the current CPU temperature?
Agent: The CPU temperature is currently 33°C.

User: Turn on the blue LED.
Agent: [turns blue LED on]

User: Turn on the yellow LED.
Agent: I'm sorry, the board does not have a yellow LED.

Configuration

Credentials

Option A: Use a LiveKit Sandbox to get up and running quickly. Setup a LiveKit Sandbox from your Cloud Project, and use its ID in your configuration:

CONFIG_LK_EXAMPLE_USE_SANDBOX=y
CONFIG_LK_EXAMPLE_SANDBOX_ID="my-project-xxxxxx"

Option B: Specify a server URL and pregenerated token:

CONFIG_LK_EXAMPLE_USE_PREGENERATED=y
CONFIG_LK_EXAMPLE_TOKEN="your-jwt-token"
CONFIG_LK_EXAMPLE_SERVER_URL="ws://localhost:7880"

Network

Connect using WiFi as follows:

CONFIG_LK_EXAMPLE_USE_WIFI=y
CONFIG_LK_EXAMPLE_WIFI_SSID="<your SSID>"
CONFIG_LK_EXAMPLE_WIFI_PASSWORD="<your password>"

Or using Ethernet (ESP32-P4 only):

CONFIG_LK_EXAMPLE_USE_ETHERNET=y

Development Board

By default, this example targets the ESP32-S3-Korvo-2 development board, using its corresponding board support package (BSP) to access the LED peripherals for the agent to control. If you wish to target a different board, this dependency can be easily removed or replaced.

If using a board other than the ESP32-S3-Korvo-2, note that this example uses the Espressif codec_board component to access board-specific peripherals for media capture and rendering. Supported boards are defined here. Locate the name of your board, and set it as follows:

CONFIG_LK_EXAMPLE_CODEC_BOARD_TYPE="S3_Korvo_V2"

Build & Flash

Navigate to this directory in your terminal. Run the following command to build your application, flash it to your board, and monitor serial output:

idf.py flash monitor

Once running, the example will establish a network connection, connect to a LiveKit room, and print the following message:

I (19508) livekit_example: Room state: Connected

If you are using the provided Sandbox, you should be able to converse with the agent at this point. Start by asking "What's the CPU temperature?"

Next Steps

Explore how the agent is built (see its source in the ./agent directory). If you are unfamiliar with LiveKit Agents, refer to the Voice AI Quickstart to learn how you can build upon the example agent or create your own from scratch.