Interactive rendering

The viewer renders novel camera views from a fixed source frame. It supports THuman and individual ZJU frames, with orbit, pan, zoom, camera reset and image export.

Install the GUI dependencies

After the CUDA environment setup, install the GUI extra:

source .cache/activate_cuda.fish
uv sync --extra gui --extra zju

Omit --extra zju for THuman only. The viewer requires a graphical desktop with OpenGL 3.3. The benchmark command runs without a display.

Launch

Select an available GPU and register data and checkpoints, then run:

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python -m eghumannerf.gui \
    --checkpoint models/thu_fullsrdf.ckpt \
    --output outputs/viewer_thuman

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python -m eghumannerf.gui \
    --experiment zju_val --checkpoint models/zju_nodepth.ckpt \
    --index 0 --output outputs/viewer_zju

Drag with the left mouse button over the image to orbit, drag with the right button to pan, and scroll to zoom. Sliders provide the same camera controls. Reset camera returns to the dataset camera. Save image exports the current rendering to the output directory.

Choose a person or source frame under Source, then click Load frame. Use --subjects to select THuman subjects or --all-subjects to browse the full validation split. The ZJU selector lists subjects and frame numbers.

The FPS display measures complete viewer iterations, including rendering, image transfer, UI drawing and buffer swap. Vertical synchronization is disabled. Window-system scheduling and display refresh can still affect presentation.

Benchmark the render loop

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python -m eghumannerf.benchmark \
    --checkpoint models/thu_fullsrdf.ckpt --subjects 401 \
    --warmup 10 --frames 100 --output outputs/benchmark_thuman.json

Add --experiment zju_val --checkpoint models/zju_nodepth.ckpt in place of the THuman checkpoint arguments for ZJU. --index selects the dataset entry.

The benchmark prepares one source frame, warms up the model, then continuously changes the target camera. Each measured frame includes camera-dependent geometry features, shell depth rasterization, ray sampling and neural rendering. The JSON report contains individual stage times, mean/median/p95 frame time and aggregate FPS. Loading, source-image encoding and display are excluded. Use a new report path for each run.

Performance options

Both the viewer and benchmark accept these options:

Option	Behavior
--rasterizer cuda	Use nvdiffrast for shell depth rasterization
--cuda-graph	Replay fixed-shape vertex feature fusion and image refinement with CUDA Graphs
--amp-refine	Use FP16 autocast in image refinement
--amp-neural	Use BF16 autocast in NeRF inference

The default uses PyTorch3D and FP32. Mixed precision and CUDA rasterization can change numerical results and edge coverage.

Install the CUDA rasterizer extra with the prepared environment:

source .cache/activate_cuda.fish
uv sync --extra gui --extra zju --extra cuda-rasterizer

For a new environment, add --with-gui --with-cuda-rasterizer to the bootstrap command. The CUDA rasterizer compiles on first use.

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python -m eghumannerf.gui \
    --checkpoint models/thu_fullsrdf.ckpt \
    --rasterizer cuda --cuda-graph --amp-refine --amp-neural \
    --output outputs/viewer_fast

Remote Linux desktops

A headless GPU server can run the viewer using VirtualGL with a TurboVNC desktop. Connect to that desktop with TurboVNC Viewer on Windows, macOS or Linux. The model and OpenGL rendering run on the server.

After preparing the remote desktop, run this command in its session:

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python scripts/run_virtualgl.py \
    --egl-device egl0 --checkpoint models/thu_fullsrdf.ckpt \
    --rasterizer cuda --cuda-graph --amp-refine --amp-neural

Select the same physical GPU for CUDA and EGL; their device indices can differ. DISPLAY must identify the remote desktop. The launcher also accepts --vglrun and repeatable --library-path arguments for a user-local VirtualGL installation. Remote delivery adds image encoding and network latency; measure client presentation separately from the server's render loop.

Record a demo

On an X11 desktop, install FFmpeg with x11grab and libx264, then add --record to a viewer command. --demo-seconds animates the camera and closes the viewer after the orbit, with a one-second hold at each end:

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python -m eghumannerf.gui \
    --checkpoint models/thu_fullsrdf.ckpt --subjects 448 \
    --rasterizer cuda --cuda-graph --amp-refine --amp-neural \
    --demo-seconds 10 --record outputs/demo/448.mp4 --output outputs/demo/448

--demo-angle sets the total orbit in radians; the default is one full turn. --demo-distance sets the camera distance scale. Recording starts after model loading and captures the complete window at wall-clock speed. The MP4 uses a 30 Hz capture stream; the GUI displays the measured render-loop FPS. Keep the window visible while recording.

To record the example sequence, run:

CUDA_VISIBLE_DEVICES=0 uv run --no-sync python scripts/record_gui_demo.py \
    --shots config/demo/gui.json --output outputs/gui_demo

For a VirtualGL desktop, add --egl-device egl0 and any --vglrun or --library-path options needed by that installation. The script records shots sequentially and joins them into outputs/gui_demo/gui-demo.mp4. The shot list contains ordinary viewer arguments and can be edited to choose other subjects, source frames or camera paths. Use a new output directory for each recording.