SurfelWarp: Efficient Non-Volumetric Dynamic Reconstruction
SurfelWarp is a dynamic reconstruction pipeline. Compared to other dynamic reconstruction methods like DynamicFusion, surfelwarp uses flat surfel array (instead of volumetric field) as the geometry representation, which makes the pipeline robust and efficient. The approach is described in our paper.
Wei Gao and Russ Tedrake, "SurfelWarp: Efficient Non-Volumetic Single View Dynamic Reconstruction", Robotics: Science and Systems (RSS) 2018 [Project][Paper][Presentation]
The code was originally developed with
CUDA 9and
PCL 1.8on Visual Studio 2015 and Ubuntu 16.04. Thanks to the contribution by @Algomorph, the code works with higher version of
CUDA, Ubuntu 18.04 and Visual Studio 2017. Also note that, for some unknown reason, the code runs much slower on Ubuntu (seems to be problem with GPU driver that only permits Debug mode).
According to your environment, please follow the specific build instruction:
CUDA 9(note that
CUDA 10doesn't work with this one)
CUDA >= 9(
CUDA 10is OK) and
cilantro
CUDA 9(
CUDA 10doesn't work with this one)
CUDA >= 9(
CUDA 10is OK) and
cilantro
We also provide a pre-built binary for the windows platform (The CUDA
-archflag for this executable is
sm_60).
We use the VolumeDeform dataset to illustrate how to run the code. An example configuration file is provided here for the "boxing" sequence. First, you need to download the boxing sequence from the VolumeDeform dataset and extract it to
data_root, your file structure should look like
${data_root}/frame-000000.color.png ${data_root}/frame-000000.depth.png ...
You also need to download the trained model for Global Patch Collider (gpc) from here. Let the path to the model be
${gpc_path}.
In the configuration file, please modify the
data_prefixand
gpc_model_pathto
${data_root}and
${gpc_path}, respectively. After that, you can run the algorithm with
cd ${project_root}/build/apps/surfelwarp_app ./surfelwarp_app /path/to/config
If everything goes well, the executable would produce the reconstructed result per frame in the same folder as
surfelwarp_app.
On the windows platform with a Nvidia Titan Xp GPU, the processing time is usually less than 20 [ms] per frame. To test the speed in surfelwarp_app, you need to build with Release and disable offline rendering (which takes most of the time).
You might override the FetchInterface and use it with ImageProcessor. If you don't have performance requirement, you can also convert your data into the format of VolumeDeform.
Currently, only periodic reinitialization is implemented. To use it, set this flag in config file. More advanced criterion of reinitialization would come soon.
The code is re-factored and improved from the repo of our RSS paper. There are some planned new features and some old code need to be ported into this repository. Here is a list of TODOs:
If you have any question or suggestion regarding this work, please send me an email.