by nkolot

nkolot / SPIN

Repository for the paper "Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the L...

316 Stars 72 Forks Last release: Not found Other 37 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

SPIN - SMPL oPtimization IN the loop

Code repository for the paper:
Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop
Nikos Kolotouros*, Georgios Pavlakos*, Michael J. Black, Kostas Daniilidis
ICCV 2019
[paper] [project page]


Installation instructions

We suggest to use the docker image we provide that has all dependencies compiled and preinstalled. Alternatively you can create a

virtual environment and install all the relevant dependencies as follows:
virtualenv spin -p python3
source spin/bin/activate
pip install -U pip
pip install -r requirements.txt

If you choose to use a virtual environment, please look at the instructions for installing pyrender.

After finishing with the installation, you can continue with running the demo/evaluation/training code. In case you want to evaluate our approach on Human3.6M, you also need to manually install the pycdf package of the spacepy library to process some of the original files. If you face difficulties with the installation, you can find more elaborate instructions here.

Fetch data

We provide a script to fetch the necessary data for training and evaluation. You need to run:

The GMM prior is trained and provided by the original SMPLify work, while the implementation of the GMM prior function follows the SMPLify-X work. Please respect the license of the respective works.

Besides these files, you also need to download the SMPL model. You will need the neutral model for training and running the demo code, while the male and female models will be necessary for evaluation on the 3DPW dataset. Please go to the websites for the corresponding projects and register to get access to the downloads section. In case you need to convert the models to be compatible with python3, please follow the instructions here.

Final fits

We also release the improved fits that our method produced at the end of SPIN training. You can download them from here. Each .npz file contains the pose and shape parameters of the SMPL model for the training examples, following the order of the training .npz files. For each example, a flag is also included, indicating whether the quality of the fit is acceptable for training (following an automatic heuristic based on the joints reprojection error).

Run demo code

To run our method, you need a bounding box around the person. The person needs to be centered inside the bounding box and the bounding box should be relatively tight. You can either supply the bounding box directly or provide an OpenPose detection file. In the latter case we infer the bounding box from the detections.

In summary, we provide 3 different ways to use our demo code and models: 1. Provide only an input image (using

), in which case it is assumed that it is already cropped with the person centered in the image. 2. Provide an input image as before, together with the OpenPose detection .json (using
). Our code will use the detections to compute the bounding box and crop the image. 3. Provide an image and a bounding box (using
). The expected format for the json file can be seen in

Example with OpenPose detection .json

python3 demo.py --checkpoint=data/model_checkpoint.pt --img=examples/im1010.jpg --openpose=examples/im1010_openpose.json
Example with predefined Bounding Box
python3 demo.py --checkpoint=data/model_checkpoint.pt --img=examples/im1010.jpg --bbox=examples/im1010_bbox.json
Example with cropped and centered image
python3 demo.py --checkpoint=data/model_checkpoint.pt --img=examples/im1010.jpg

Running the previous command will save the results in

. The file
shows the overlayed reconstruction of the model on the image. We also render a side view, saved in

Run evaluation code

Besides the demo code, we also provide code to evaluate our models on the datasets we employ for our empirical evaluation. Before continuing, please make sure that you follow the details for data preprocessing.

Example usage:

python3 eval.py --checkpoint=data/model_checkpoint.pt --dataset=h36m-p1 --log_freq=20
Running the above command will compute the MPJPE and Reconstruction Error on the Human3.6M dataset (Protocol I). The
option can take different values based on the type of evaluation you want to perform: 1. Human3.6M Protocol 1
2. Human3.6M Protocol 2
3. 3DPW
4. LSP

You can also save the results (predicted SMPL parameters, camera and 3D pose) in a .npz file using


For the MPI-INF-3DHP dataset specifically, we include evaluation code only for MPJPE (before and after alignment). If you want to evaluate on all metrics reported in the paper you should use the official MATLAB test code provided with the dataset together with the saved detections.

Run training code

Due to license limitiations, we cannot provide the SMPL parameters for Human3.6M (recovered using MoSh). Even if you do not have access to these parameters, you can still use our training code using data from the other datasets. Again, make sure that you follow the details for data preprocessing.

Example usage:

python3 train.py --name train_example --pretrained_checkpoint=data/model_checkpoint.pt --run_smplify
You can view the full list of command line options by running
python3 train.py --help
. The default values are the ones used to train the models in the paper. Running the above command will start the training process. It will also create the folders
that are used to save model checkpoints and Tensorboard logs. If you start a Tensborboard instance pointing at the directory
you should be able to look at the logs stored during training.


If you find this code useful for your research or the use data generated by our method, please consider citing the following paper:

  Title          = {Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop},
  Author         = {Kolotouros, Nikos and Pavlakos, Georgios and Black, Michael J and Daniilidis, Kostas},
  Booktitle      = {ICCV},
  Year           = {2019}

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.