HAVC Client
July 10, 2026 · View on GitHub
A FreeSimpleGUI-based desktop client that connects to the HAVC DiT Server and orchestrates a full video colorization pipeline: extraction → AI colorization → encoding → merge.
Table of Contents
- Pipeline Overview
- System Requirements
- Installation
- Launching the GUI
- Interface Guide
- Workflow: Step by Step
- Understanding the Merge Step
- Settings Persistence
- Credits
Pipeline Overview
Original Video
│
▼
Step 1: EXTRACT
(VapourSynth → vscmnet2 → reference frames in ref_tht10/)
│
▼
Step 2: COLORIZE
(RPC → HAVC DiT Server → colorized frames in ref_qwen/)
│
▼
Step 3: ENCODE
(VapourSynth → vscmnet2 → x265 or NVEnc → .h265 video)
│
▼
Step 4: MERGE (optional)
(VapourSynth → vscmnet2 → blended .h265 → .mkv)
Each step can be toggled on/off independently from the Dashboard. For example, if reference frames are already extracted you can skip Step 1 and run only Steps 2–4.
System Requirements
| Requirement | Details |
|---|---|
| OS | Windows 10 / 11 (64-bit) |
| Python | 3.12 |
| GPU | RTX 30 / 40 / 50 (same as the DiT server) |
| Tools | VapourSynth R74, x265, NVEncC, MKVToolNix |
| Server | dit_rpc_server.py must be running (see main README) |
The GUI and the server can run on the same machine (localhost) or on different machines — just point the RPC host field to the server's IP.
Installation
All packages must be installed in the same .venv already created for the
DiT RPC server. If you haven't set up the server yet, follow the
main README first.
1. Activate the shared virtual environment
# From the project root (HAVCServerDiT)
.venv\Scripts\activate
2. Install GUI Python dependencies
pip install -r GUI\requirements.txt
This installs:
- Pillow — image loading and preview
- FreeSimpleGUI — GUI toolkit (PySimpleGUI fork)
- VapourSynth R74 — video frameserver bindings
- send2trash — safe file deletion
- tkinter_embed — tkinter integration for FreeSimpleGUI
3. Install vscmnet2
The vscmnet2 package provides the VapourSynth functions used by Steps 1, 3,
and 4 (scene-change detection, edge-aware frame extraction, color merging,
and encoding). It is available from github.com/dan64/vs-cmnet2:
pip install packages\vscmnet2-1.0.6-py3-none-any.whl
To complete the installation of this filter is necessary to install the models, weights and plugins, as described in the filter home page: vs-cmnet2
4. Install spatial_correlation_sampler
This is a compiled extension (PyTorch 2.10 + CUDA 13.0) of
Pytorch-Correlation-extension,
required by vscmnet2 for temporal alignment during encoding:
pip install packages\spatial_correlation_sampler-0.5.0-cp312-cp312-win_amd64.whl
The wheel is pre-built for Python 3.12 / PyTorch 2.10+cu130 / Windows x64. It will only work with that exact combination.
5. Install external tools
The GUI relies on three command-line tools that must be present on disk (they are not Python packages):
| Tool | Purpose | Default location | Download |
|---|---|---|---|
| VapourSynth | Video frameserver | Bundled in the .venv | pip install VapourSynth==74 |
| x265 | H.265 software encoder | GUI/tools/x265/x265.exe | x265 downloads |
| NVEncC | NVIDIA GPU encoder | GUI/tools/NVEncC/NVEncC64.exe | rigaya/NVEnc |
| MKVToolNix | .h265 → .mkv muxing | GUI/tools/MKVToolNix/mkvmerge.exe | MKVToolNix |
Quick setup with Release 1.0.0: the project's Release 1.0.0 includes a
tools.ziparchive containingx265.exeandmkvmerge.exe. Download it and extract its contents directly intoGUI/tools/so that the default paths match without any additional configuration:GUI/tools/ ├── x265/ │ └── x265.exe ├── MKVToolNix/ │ └── mkvmerge.exe ├── NVEncC/ (download separately) └── ...You can also place these tools anywhere — just point the GUI to their paths in the Encode/Merge tab.
Launching the GUI
From the command line
# Activate the venv first
.venv\Scripts\activate
python GUI\CMNET2_colorize_client_GUI.py
From the launcher (silent, no console)
Double-click GUI/run_colorize_client_GUI.vbs. This runs the .cmd wrapper
in a hidden console, so no Terminal window stays open.
Desktop shortcut (optional)
You can create a desktop shortcut to launch the GUI without ever seeing a command prompt:
-
Right-click on the desktop → New → Shortcut
-
For the location, enter the full path to the
.vbsfile:D:\PProjects\HAVCServerDiT\GUI\run_colorize_client_GUI.vbs -
Click Next, give the shortcut a name (e.g. CMNET2 Colorize Client)
-
Click Finish
To change the shortcut's icon:
- Right-click the new shortcut → Properties
- Click Change Icon...
- Browse to any
.icofile on your system (or download one you like) - Click OK twice
The shortcut launches the GUI silently – the VBScript runs the .cmd wrapper
in a hidden window.
Before using .cmd / .vbs launcher
The launcher auto-detects the Python interpreter in this order:
- Explicit
PYTHON_EXE– set it inGUI/run_colorize_client_GUI.cmdif you use a custom environment .venvin the project root – looks for.venv\Scripts\python.exeone level aboveGUI/pythonfrom PATH – fallback
If you use a non-standard environment location, edit PYTHON_EXE in the
USER CONFIGURATION block at the top of run_colorize_client_GUI.cmd:
set PYTHON_EXE=C:\Users\YourName\.conda\envs\my-env\python.exe
Interface Guide
The GUI has seven tabs plus a persistent status bar at the bottom.
Dashboard
- Task checkboxes: enable/disable each pipeline step
- START PIPELINE: runs the selected steps sequentially
- STOP: gracefully interrupts the current step (sends stop signal to both subprocesses and the RPC server)
- Progress bar: shows overall completion percentage
- Log window: live output from VapourSynth, x265/NVEnc, and the RPC client
- Shutdown PC when finished: triggers
shutdown /s /t 60after completion
Tab 1 — Extraction
| Setting | Description |
|---|---|
| VapourSynth Pipe | Path to vspipe.exe (bundled in .venv) |
| Script Directory | Folder containing the .vpy scripts (GUI/scripts/) |
| Extract VPY | VapourSynth script for frame extraction |
| Threshold / tht_ssim / min_int / mult/freq | Scene-change detection parameters |
| Ref Override | Force re-extraction even if reference frames exist |
| Duplicate first frame | Copies the second extracted frame to ref_000000.jpg (useful for frame 0 coverage) |
| Video Directory | Folder containing the video to process |
| Select Video | Dropdown populated from the video directory |
After selecting a video, the Video Technical Details panel shows resolution, FPS, frame count, and pixel format.
Tab 2 — Colorization
| Setting | Description |
|---|---|
| RPC Host / Port | Server address (default: 127.0.0.1:8765) |
| Connect button + LED | Tests the RPC connection with a ping |
| Model / Precision | Pipeline configuration. Precision selects the GGUF quant level (q3–q8) or Nunchaku variant (fp4/int4). LongCat GGUF available via longcat-gguf model |
| Run Server | Launch start_server.cmd with the selected model + precision in a new terminal window |
| Colorization Steps | Diffusion steps per frame (lower = faster). LongCat recommends 8 steps, Qwen 2 steps |
| Fast Pipeline | Enables paired inference: two frames colorized in one forward pass (~2× faster, temporally consistent). Only supported by nunchaku-qwen |
| Prompt | Text prompt sent to the model |
| Cache Directory | HuggingFace cache (leave empty for default) |
The two image panels show a live preview of the B&W input and the AI output as frames are processed.
Tab 3 — Encode / Merge
| Setting | Description |
|---|---|
| MKVmerge Path | Path to mkvmerge.exe |
| x265 Path | Path to x265.exe (also used to locate NVEncC) |
| Encode VPY | VapourSynth script for encoding |
| CRF | x265 quality (lower = better, typical: 18–24) |
| FPS | Output frame rate |
| Encoder | x265 (software) or Nvenc (GPU hardware) |
| Memory Frames | Max frames buffered by VapourSynth |
| Render Speed | VapourSynth render preset (auto, fast, medium, slow, slower) |
| Merge Weight | Blend ratio for Step 4 (0.30 = 30% original, 0.75 = 75% original) |
| VBR Quality | NVEnc quality target (lower = better) |
| NVEnc Sharpness | Enables --vpp-unsharp --vpp-edgelevel on NVEnc |
Tab 4 — Fix Image
A standalone image colorization tab independent of the video pipeline. Supports both SHM (same-host) and PNG-over-RPC (remote server) transport.
Single‑image mode (default):
| Control | Description |
|---|---|
| Colorization Steps | Inference steps (default: 2) |
| Convert in B&W | Convert the input to grayscale before colorization (useful for re‑colorizing) |
| Prompt | Text prompt for the model (combo with history) |
| Max / Delete / Clear | Prompt history management |
| Load Image | Load image via the ComboBox, drag‑and‑drop, or Browse |
| Colorize | Run colorization with fixed seed (42) |
| Colorize (Random) | Run colorization with random seed for variation |
| Overwrite | Overwrite the original loaded image with the colorized result |
| Save As... | Save the colorized result (PNG / JPG) |
| Swap Output → Input | Copy the output image as input for the next colorization |
Batch processing (Enable batch processing checkbox):
| Control | Description |
|---|---|
| Enable batch processing | Toggle between single‑image and batch mode |
| Clear | Remove all images from the batch list |
| ComboBox | Shows all loaded images (by basename); select one to preview |
| Overwrite | Overwrites all original files with their colorized versions |
| Save As... | Shows a mask dialog (*_colorized.png); * is replaced with each filename |
| Swap Output | Disabled in batch mode |
In batch mode, each loaded image (drag‑and‑drop or Browse) is appended to the
list. Pressing Colorize processes all images sequentially: the current
input is shown in the preview, the output updates live, and the status bar
shows progress (✅ 3/7). Outputs are kept in memory — files are written to
disk only on Overwrite or Save As. Errors on individual images are
skipped automatically.
Prerequisite: the HAVC DiT Server must be connected (Tab 2 — Connect).
Tab 5 — Fix Colors
A standalone image colorization tab that uses the local CMNET2 model (exemplar-based color propagation) instead of the DiT RPC server. It colorizes a B&W target image using a color reference image as context.
Single‑image mode (default):
| Control | Description |
|---|---|
| Reference Image (Color) | Load a color reference image (drag & drop or Browse) that provides the color palette |
| Target Image (B&W) | Load the B&W image to colorize via ComboBox, drag & drop, or Browse |
| Colorize | Run CMNET2 colorization in a background thread. The first call loads the model (~5–10 s); subsequent calls reuse it (~1–2 s) |
| Overwrite | Overwrite the original target file with the colorized result |
| Save As... | Save the colorized result to a new file (PNG / JPG) |
| Copy → Fix Image | Copy the colorized output as the input image for Tab 4 (Fix Image), enabling a two‑stage pipeline: CMNET2 → DiT RPC |
Batch processing (Enable batch processing on the Target Image):
| Control | Description |
|---|---|
| Enable batch processing | Toggle batch mode for the Target Image |
| Clear | Remove all target images from the batch list |
| ComboBox | Shows all loaded target images; select one to preview |
| Overwrite | Overwrites all original target files with their colorized versions |
| Save As... | Mask dialog (*_colorized.png); * replaced with each filename |
| Copy → Fix Image | Disabled in batch mode |
In batch mode, a single reference image colorizes multiple target images
sequentially. Each target is added via drag‑and‑drop or Browse. Colorize
processes all targets against the same reference, showing progress
(✅ 3/7). Outputs stay in memory until Overwrite or Save As is
clicked.
Three preview panels show the reference, target, and colorized output side‑by‑side (250×240 px each). Full‑resolution images are always preserved in memory; resizing only applies to the previews.
Prerequisite:
vscmnet2must be installed and model weights/checkpoints must be present (see Installation). No RPC connection required — CMNET2 runs entirely on the local GPU.
Tab 6 — Fix Video
A standalone video recoloring tab that runs a VapourSynth + NVEnc pipeline using the selected video, encode script, and two reference images.
| Control | Description |
|---|---|
| Video Directory | Folder containing the video to recolor |
| Select Video | Dropdown populated from the video directory |
| Encode VPY | VapourSynth script for encoding / recoloring (script: encode_cmnet2_recolor.vpy) |
| FPS | Output frame rate |
| VBR Quality | NVEnc quality target (lower = better) |
| Memory Frames | Max frames buffered by VapourSynth |
| Render Speed | VapourSynth render preset (auto, fast, medium, slow, slower) |
| First Reference | Load a reference image (drag & drop or Browse) for the start of the clip |
| Last Reference | Load a reference image (drag & drop or Browse) for the end of the clip |
| Recolor | Runs the VapourSynth → NVEnc pipeline in a background thread |
The two reference images guide the recoloring script: they are passed to the
VapourSynth script as RefStart and RefEnd parameters. The RefDir
parameter is automatically set to the folder containing the first reference
image. Only the frames between RefStart / RefEnd will be recolored.
Startup check: before launching, the tab verifies that NVEncC64.exe exists
next to x265.exe (derived from the x265 path configured in Tab 3). If
missing, an error popup instructs the user to install NVEncC in tools\NVEncC.
The output file is named [video]_cmnet2_dt-recolor.mkv and is saved in the
video directory. The .h265 intermediate is automatically converted to .mkv
via mkvmerge and deleted. A log file [video]_dt-recolor_log.txt (or
_cmnet2_dt-recolor_log.txt) is also written to the same directory with the
full session output.
Important: this tab uses NVEnc only — the x265 software encoder is not available here. NVEncC64.exe must be installed.
Workflow: Step by Step
Step 1: Extract Reference Frames
VapourSynth reads the video through vscmnet2 and runs scene-change
detection (sc_algo=1). Keyframes that differ significantly from their
neighbors are exported as JPEG images to ref_tht10/.
The extraction parameters control how aggressively frames are selected:
- threshold (
sc_threshold): sensitivity — lower = more frames - tht_ssim (
sc_tht_ssim): SSIM threshold for scene-change - min_int (
sc_min_int): minimum frame interval between selections - mult/freq (
sc_mult_tht): minimum frequency multiplier
Step 2: Colorize Frames (AI)
The GUI connects to the HAVC DiT Server and sends each extracted frame for
colorization. Results are saved to ref_qwen/.
Two modes are available:
- Standard: each frame is sent individually
- Fast Pipeline (paired inference): two frames are placed side-by-side and colorized in a single forward pass. This is faster and produces temporally-consistent results between adjacent frames.
The server's pipeline is loaded on demand (if not already loaded at boot).
Step 3: Encode Video
VapourSynth reads the original video and the colorized frames from
ref_qwen/, then vscmnet2 overlays the color onto the original luminance
channel. The result is piped to the chosen encoder.
| Encoder | Pros | Cons |
|---|---|---|
| x265 | Higher quality, fine CRF control | Slower (CPU-bound) |
| NVEnc | Fast (GPU), VBR quality control | Requires NVIDIA GPU |
The output is a .h265 raw video stream. If MKVToolNix is configured, a
.mkv container is created automatically and the raw .h265 is deleted.
Step 4: Merge (optional)
This step only makes sense when the original video clip is already colorized (e.g., a previous colorization pass or a naturally color source).
VapourSynth reads both the original color clip and the newly encoded DiT
clip, then blends them using vscmnet2.vs_merge:
$ \text{output} = (\text{DiT} \text{clip} \times (1 - \text{weight})) + (\text{original} \text{color} \text{clip} \times \text{weight}) $
The Merge Weight slider (0.30–0.74) controls how much of the original color clip is kept:
| Weight | Effect |
|---|---|
| 0.30 | 30% original color, 70% DiT — DiT look dominates |
| 0.50 | 50/50 — balanced blend |
| 0.74 | 74% original color, 26% DiT — original colors dominate |
If the original clip is black-and-white, disable Step 4. Blending a B&W clip with a colorized one only desaturates the result.
The merged output is saved as [video]_cmnet2_dt-color_merged.mkv.
Understanding the Merge Step
The merge is a luminance-guided chroma blend: the vscmnet2.vs_merge
function takes the chroma from the DiT-colorized clip and the chroma from the
original color clip, then blends them according to the weight parameter while
keeping the original luma channel.
This is useful when:
- You have a manually colorized version of the clip and you want to see which parts the AI interprets differently
- You want to tone down the AI's color choices by mixing in a known-good reference
- You want to compare the AI output against a baseline by creating a 50/50 split
Settings Persistence
All GUI settings are saved to gui_cmnet2_settings.json in the GUI/ folder
when you click Save Global Settings. The file is loaded automatically at
startup and includes:
- Tool paths (VapourSynth, x265, MKVToolNix)
- Script directory and script filenames
- Model configuration (name, precision, rank, steps)
- Extraction parameters
- Encoding parameters (CRF, FPS, encoder choice)
- Merge weight and VBR quality
- Fix Colors reference/target image paths
- RPC host and port
- Window size
Credits
- CMNET2 / vscmnet2: github.com/dan64/vs-cmnet2 — VapourSynth color-matching and scene-detection functions
- spatial_correlation_sampler: Pytorch-Correlation-extension — GPU correlation layer used by vscmnet2
- DiT Model: Qwen/Qwen-Image-Edit-2511, LongCat-Image-Edit-Turbo
- VapourSynth: vapoursynth.com
- x265: videolan.org
- NVEncC: rigaya/NVEnc
- MKVToolNix: mkvtoolnix.download