K-Planes nerfstudio integration

This repository provides code to integrate the K-Planes model into nerfstudio.

It provides an alternative way to use k-planes in addition to the official repository, which allows access to nerfstudio's in-browser viewer and additional training capabilities. Beware that some details about the training procedure differ from the official repository.

Installation

1. Install nerfstudio. This is pip install nerfstudio, but there are a few dependencies (e.g. torch, tinycudann) which may require further steps, so make sure to check their installation guide!
2. Install the k-planes nerfstudio integration (this repository): pip install git+https://github.com/Giodiro/kplanes_nerfstudio

Included Models

Two models are included here:

• kplanes which is tuned for the Synthetic NeRF dataset (i.e. chair, drums, etc.)
• kplanes-dynamic which is tuned to the DNeRF dataset (dynamic, monocular video).

:exclamation: PRs are welcome for configurations tuned to different datasets :exclamation:

You can run the static model by calling (remember to use the correct data directory!)

ns-train kplanes --data <data-folder>

and connect to the viewer using the link provided in the output of the training script.

Benchmarks

Synthetic NeRF (hybrid model)

	drums	materials	ficus	ship	mic	chair	lego	hotdog	AVG
PSNR	26.31	29.82	32.47	30.27	33.73	34.98	36.56	36.77	32.61
SSIM	0.9394	0.9539	0.9788	0.8755	0.9857	0.9824	0.982	0.9792	0.9596

DNeRF (hybrid model)

	hell warrior	mutant	hook	balls	lego	t-rex	stand up	jumping jacks	AVG
PSNR	25.06	34.29	28.22	43.02	27.03	33.59	34.04	33.43	32.33
SSIM	0.9487	0.9839	0.9552	0.9954	0.956	0.9817	0.9835	0.9797	0.973

Roadmap

Expected future updates to this repository:

• Including all datasets used in the K-Planes paper
• Clarifying configuration of colliders (near-far)
• Add benchmarks and configs for linear models