Exercise4.md

July 17, 2026 · View on GitHub

Exercise 4 - Full open source DMF

Synopsis

In this exercise, we will compile the latest commit of the MXL SDK including rust bindings and the rust Gstreamer plugins. Then we will build a full stream augmentation workflow supported by various open source project. A full list is found at the end of this document.

    flowchart LR
        subgraph HostNode [Host / Compute Node]
            direction LR

            %% --- Stage 1: Sources ---
            subgraph Sources [Sources]
                direction TB
                HLS2MXL[HLS2MXL]
                LoopPlayer[File Player]
                TestGen[Test Generator]
                WebRTC2MXL[WebRTC2MXL]
            end

            %% --- Stage 2: Processing ---
            subgraph Processing [Processing]
                direction TB
                AudioMix[Audio Mix]
                HTML5Keyer[HTML5 Keyer]
                SPXGraphics[SPX Graphics]
            end

            %% --- Stage 3: Output ---
            MXL2WebRTC[MXL2WebRTC]

            %% =========================================
            %% CONNECTIONS
            %% =========================================

            %% --- Solid Video (blue) Connections (Indices 0 to 4) ---
            %% Video Sources to Input selector
            HLS2MXL -- IN 1 --> InputSel
            LoopPlayer -- IN 2--> InputSel
            TestGen -- IN 3 --> InputSel
            

            %% Input Selector to Processing
            InputSel --> HTML5Keyer
            
            %% Processing to Output
            HTML5Keyer --> MXL2WebRTC

            %% --- Solid Audio (green) Connection (Indices 5 to 7) ---
            %% Audio Sources to Audio Mixer
            WebRTC2MXL -- IN 2 --> AudioMix
            HLS2MXL -- IN 1 --> AudioMix

            %% Audio Processing to/From MXL
            AudioMix --> MXL2WebRTC

        end

        %% --- Legend (Outside the Compute Node) (Indices 8 to 10) ---
    subgraph Legend [Diagram Key]
        direction LR
        k1[Gstreamer</br>based app] -- MXL Video flow --> kt1[Gstreamer</br>based app]
        k2[Gstreamer</br>based app] -- MXL Audio flow --> kt2[Gstreamer</br>based app]
        k4[HTML 5 Graphic</br>engine]
        k5[Future Apps]
    end

    %% Positioning the legend below the main node
    HostNode ~~~ Legend

        %% ===========================================
        %% STYLING
        %% ===========================================

        %% Node Styling
        classDef gstreamer fill:#66b3ff,color:black,stroke:#333,stroke-width:2px;
        classDef control fill:#007bff,color:#fff,stroke:#333,stroke-width:2px;
        classDef other fill:#cce6ff,color:black,stroke:#333,stroke-width:2px;
        classDef future fill:#ffcccc,color:black,stroke:#cc0000,stroke-width:2px,stroke-dasharray:5 5

        class HLS2MXL,LoopPlayer,TestGen,InputSel,HTML5Keyer,MXL2WebRTC,WebRTC2MXL,k1,kt1,k2,kt2,k3 gstreamer
        class AudioMix,k5 future
        class NmosRegistry,DummyNmosNode,NmosController,kt3 control
        class SPXGraphics,k4 other

        %% Link Styling
        %% Solid blue for video connections
        linkStyle 0,1,2,3,4,8 stroke:blue,stroke-width:2px

        %% Solid green for audio connections
        linkStyle 5,6,7,9 stroke:green,stroke-width:2px

Steps

  1. Navigate to exercise 5 working directory

        cd ~/mxl-hands-on/docker/exercise-4

  2. If you did NOT do the preparations steps for WLS, make sure you have a /Volumes/mxl mounted in tmpfs.

    sudo mount -t tmpfs -o size=512m,uid=1000,gid=1000,mode=0755 tmpfs /Volumes/mxl # on WSL linux
sudo mkdir -p /Volumes/mxl/domain_1
sudo chown 1000:1000 /Volumes/mxl/domain_1

  3. Making sure we have at least one clip for our clip player application (you can add more of your own, .mp4 or .ts)

    cp ~/mxl-hands-on/build-images/sizzle.ts data/Clips

  4. Start the system with the start script.

        ./start.sh # For linux based system

        ./start-mac.sh # For mac based system

  5. Use the application and try to reproduce the workflow above. You have more documentation on application usage here

  6. When you are down, do not forget to shutdown your containers.

    stop.sh # For linux based system

    ./stop-mac.sh # For mac based system
AppURLAPI Swagger Page
Test Generatorhttp://localhost:9600http://localhost:9600/docs
MXL Info GUIhttp://localhost:9699http://localhost:9699/docs
MXL to WebRTChttp://localhost:9601http://localhost:9601/docs
File Playerhttp://localhost:9602http://localhost:9602/docs
HLS to MXL Gatewayhttp://localhost:9603http://localhost:9603/docs
Input Selectorhttp://localhost:9604http://localhost:9604/docs
HTML5 Keyerhttp://localhost:9605http://localhost:9605/docs
SPX serverhttp://localhost:5660none

Reference HLS stream that are 1920x1080p60

    https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8

    https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8

Open Source Components

ComponentCategoryLicenseTest GenInfo GUIMXL2WebRTCFile PlayerHLS2MXLInput SelHTML5 Keyer
ReactFrontendMIT✓✓✓✓✓✓✓
React DOMFrontendMIT✓✓✓✓✓✓✓
ViteFrontendMIT✓✓✓✓✓✓✓
@vitejs/plugin-reactFrontendMIT✓✓✓✓✓✓✓
MXLMXLApache-2.0✓✓✓✓✓✓✓
FastAPIPython 3MIT✓✓✓✓✓✓✓
UvicornPython 3BSD-3-Clause✓✓✓✓✓✓✓
aiofilesPython 3Apache-2.0✓✓✓✓✓✓✓
PydanticPython 3MIT——✓—✓✓✓
python-multipartPython 3Apache-2.0✓✓—✓✓——
RequestsPython 3Apache-2.0✓———✓——
PyGObjectGStreamerLGPL-2.1+✓—✓✓✓✓✓
GStreamerGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer plugins-baseGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer plugins-goodGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer plugins-badGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer plugins-uglyGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer libavGStreamerLGPL-2.0+✓—✓✓✓✓✓
GStreamer nice (libnice)GStreamerLGPL-2.1——✓————
gstcefsrcGStreamerLGPL-2.0+——————✓
CEF (Chromium Embedded Framework)Browser EngineBSD-3-Clause——————✓
MediaMTXInfrastructureMIT——✓————
Ubuntu 24.04Base ImageVarious✓✓✓✓✓✓✓
Node.js 18BuildMIT✓✓✓✓✓✓✓