PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing

WACV 2025

Project Page | Paper

Siddharth Seth¹, Rishabh Dabral², Diogo Luvizon², Marc Habermann², Ming-Hsuan Yang¹, Christian Theobalt², Adam Kortylewski^2,3
1UC Merced, ²Max Planck Institute for Informatics, ³University of Freiburg

Abstract (click to view)

Modeling a human avatar that can plausibly deform to articulations is an active area of research. We present POCOLOCO – the first template-free, point-based, pose-conditioned generative model for 3D humans in loose clothing. We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations. Consequently, they are restricted to modeling clothing that is topologically similar to the naked body and do not extend well to loose clothing. The few methods that attempt to model loose clothing typically require either canonicalization or a UV-parameterization and need to address the challenging problem of explicitly estimating correspondences for the deforming clothes. In this work, we formulate avatar clothing deformation as a conditional point-cloud generation task within the denoising diffusion framework. Crucially, our framework operates directly on unordered point clouds, eliminating the need for a parametric model or a clothing template. This also enables a variety of practical applications, such as point cloud completion and pose-based editing – important features for virtual human animation. As current datasets for human avatars in loose clothing are far too small for training diffusion models, we release a dataset of two subjects performing various poses in loose clothing with a total of 75K point clouds. By contributing towards tackling the challenging task of effectively modeling loose clothing and expanding the available data for training these models, we aim to set the stage for further innovation in digital humans.

Code setup

Anticipated release date: July 1, 2025 Released. An initial version of the training code has been released. While this is forked from the working code, it is still pending cleanup and may require minor modifications to run as intended. The code contains all the elements for training a point-based 3D avatar model - data loader, file utilities, and training.

LOOSE Dataset

To be released soon