Shading-aware Multi-view Stereo
This repository contains an implementation of 'Shading-aware Multi-view Stereo' [1] [pdf]. The framework itself is based on 'A New Variational Framework for Multiview Surface Reconstruction' [2] [pdf].
The code only depends on MVE, libjpeg, libpng, and libtiff. Compilation is supported on Unix and macOS systems:
git clone https://github.com/simonfuhrmann/mve.git git clone https://github.com/flanggut/smvs.git make -C mve make -C smvs
Before you start please have a look at the general reconstruction pipeline in the MVE Wiki. This project is intended as an alternative multi-view stereo step and replaces the
dmreconand
scene2psetapplications with the
smvsrecon. The complete pipeline is therefore:
makescene -i sfmrecon smvsrecon fssrecon /smvs-[B,S].ply /smvs-surface.ply meshclean -p10 /smvs-surface.ply /smvs-clean.ply
If you run
smvsreconwithout any arguments it automatically chooses the most robust and reasonably fast setting. The images are rescaled to be on average around 2MP, the shading-based optimization is disabled and the optimization is running up to scale 2; the output will be
smvs-B.ply. This behavior can be changed with various command line arguments.
-S. This will output the point cloud
smvs-S.ply. Keep in mind that the lighting model is limited to a low-dimensional global illumination based in spherical harmonics. As noted in the paper this model cannot handle complex scenes. Also try to supply linear images to the reconstruction pipeline that are not tone mapped or altered as this can also have very negative effects on the reconstruction. If you have simple JPGs with SRGB gamma correction you can remove it with the
--gamma-srgboption.
smvshas two scale options.
-saffects the size of the input images and will downscale them by the respective power of 2 (this is analog to the same option of
dmrecon, e.g.
-s2would downscale to 1/4th of the original size).
-oaffects the scale of the optimization - the finest resolution of the bicubic patches will have the size of the respective power of 2 (e.g.
-o2will optimize patches covering down to 4x4 pixels).
--no-sgm. Additionally, this allows for the reconstruction of scenes that have not been generated via structure-from-motion and do not contain an initial sparse point cloud. In this case the range for an initial depth sweep has to be supplied via
--sgm-range.
For more details please also have a look at the usage output of the application. Note that some of the features are still WIP.
[1] Shading-aware Multi-view Stereo - Fabian Langguth, Kalyan Sunkavalli, Sunil Hadap, Michael Goesele - ECCV 2016
@inproceedings{langguth-2016-smvs, title = {Shading-aware Multi-view Stereo}, author = {F. Langguth and K. Sunkavalli and S. Hadap and M. Goesele}, booktitle = {Proceedings of the European Conference on Computer Vision ({ECCV})}, year = {2016} }
[2] A New Variational Framework for Multiview Surface Reconstruction - Ben Semerjian - ECCV 2014
@inproceedings{semerjian-2014-varsurf, title = {A New Variational Framework for Multiview Surface Reconstruction}, author = {B. Semerjian}, booktitle = {Proceedings of the European Conference on Computer Vision ({ECCV})}, year = {2014} }