Audio Subsystem

For usage related information, see audio feature.

The prefix for all packets and capabilities is audio.

Component	Link
client	xpra.client.subsystem.audio
client connection	xpra.server.source.audio
server	xpra.server.subsystem.audio

xpra.audio contains the components used for capturing and playing back audio streams using GStreamer.
In order to avoid interfering with the performance of the main thread, all audio processing is done in a separate process.
For historical reasons, this is done using a subprocess wrapper rather than the builtin multiprocessing module.

Pulseaudio

xpra.audio.pulseaudio is often used for playback on Linux systems.
This is also the prefered backend for audio capture in server sessions. The xpra server will usually start a pulseaudio instance hidden away in a per-session user prefix so that multiple sessions can forward audio streams independently.

The client and server should expose the following capabilities in their hello packet with the audio prefix:

Capability	Type	Purpose
decoders	List of strings	The audio formats that can be received and decoded
encoders	List of strings	The audio formats that can be encoded and sent
send	boolean	If sending audio is enabled
receive	boolean	If receiving audio is enabled

The lists of decoders and encoders contain strings such as: mp3, opus+ogg, vorbis...
You can run xpra.audio.gstreamer_util to see which encoders and decoders are available on the system.

This protocol is identical in both directions. Audio being forwarded from the server to the client (aka "speaker forwarding") uses the same packets as audio coming from the client to the server (aka "microphone forwarding").

Packet Type	Arguments	Purpose	Information
audio-data	codec : string data : bytes attributes : dictionary	Audio stream data	The initial and final packets may omit the data argument and should set the start-of-stream / end-of-stream attributes
audio-control	subcommand : string (ie: start, stop, sync, new-sequence) argument : Any	Send a request to the peer
audio-capabilities	capabilities dictionary	Update audio formats and directions	Sent when asynchronous audio capability discovery completes
audio-keepalive	monotonic timestamp	Keep an active stream alive	Either direction
audio-level	sample dictionary	Publish meter values	Server to client when requested
audio-signal	boolean	Publish whether a non-silent signal is present	Server to client when requested