TopoDiT-3D

The illustration of the proposed Topology-Aware Diffusion Transformer (TopoDiT-3D) for 3D point cloud generation. TopoDiT-3D initially voxelizes the point clouds, employing the patch operator to generate tokens related to the local point-voxel feature, and the persistence images to generate tokens related to the global topological feature. The persistence images are generated by the pretrained VAE during inference. Subsequently, TopoDiT-3D uses a fixed minor number of learned queries and the Perceiver Resampler to downsample and learn the topological and geometric information. After $N$ DiT-3D blocks, it uses the Perceiver Resampler to achieve upsampling, which recovers the same number of patch tokens to devoxelize.

Useage

Requirements

• Install Environments:

python==3.6
pytorch==1.7.1
torchvision==0.8.2
cudatoolkit==11.0
matplotlib==2.2.5
tqdm==4.32.1
open3d==0.9.0
trimesh=3.7.12
scipy==1.5.1

• Install PyTorchEMD:

cd metrics/PyTorchEMD
python setup.py install
cp build/**/emd_cuda.cpython-36m-x86_64-linux-gnu.so .
# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl

• Install PyTorch_VAE:

cd topology_vae/PyTorch_VAE
pip install -r requirements.txt

• Install Persim:

pip install persim

Or please simply run

pip install -r requirements.txt

Train

# Train TopoDiT-3D
bash ./scripts_resampler/train/train_S4_chair.sh

Test

# Train Vae
bash ./scripts_resampler/train_vae.bash

# Test TopoDiT-3D
bash ./scripts_resampler/train/test_S4_chair.sh