TailorNet

This repository contains training and inference code for the following paper:

TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style  
Chaitanya Patel*, Zhouyingcheng Liao*, Gerard Pons-Moll  
CVPR 2020 (ORAL)  

Citation

Cite us if you use our model, code or data:

@inproceedings{patel20tailornet,
        title = {TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style},
        author = {Patel, Chaitanya and Liao, Zhouyingcheng and Pons-Moll, Gerard},
        booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
        month = {jun},
        organization = {{IEEE}},
        year = {2020},
    }

Updates

• [26-12-2020] Female skirt weights added.
• [11-11-2020] Female and male short-pant weights added.
• [02-08-2020] Female and male pant weights added.
• [19-07-2020] Male shirt weights added.
• [12-07-2020] Female shirt weights added.
• [28-06-2020] Female t-shirt weights added.
• [25-06-2020] Minor bug fixes and male t-shirt weights added.
• [17-06-2020] Inference script and female old-t-shirt weights added.

Requirements

python3
pytorch
scipy
chumpy
psbody.mesh

Code works with psbody.mesh v0.4 , pytorch >= v1.0 , chumpy v0.7 and scipy v1.3 .

How to Run

• Download and prepare SMPL model and TailorNet data from dataset repository.
• Set DATADIR and SMPL paths in `globalvar.py` file accordingly.
• Download trained model weights in a directory and set its path to MODELWEIGHTSPATH variable in

  global_var.py

  .
  • old-t-shirtfemaleweights (4.1 GB)
  • t-shirtmaleweights (2.0 GB)
  • t-shirtfemaleweights (2.0 GB)
  • shirtfemaleweights (2.5 GB)
  • shirtmaleweights (2.5 GB)
  • This link has all the weights listed above as well as the following:
  • pantfemaleweights
  • pantmaleweights
  • short-pantfemaleweights
  • short-pantmaleweights
  • skirtfemaleweights
• Set output path in

  run_tailornet.py

  and run it to predict garments on some random inputs. You can play with different inputs. You can also run inference on motion sequence data.
• To visualize predicted garment using blender, run

  python run_tailornet.py render

  . (Blender 2.79 needs to be installed.)

TailorNet Per-vertex Error in mm on Test Set

... evaluated using

evaluate

utils/eval.py

Training TailorNet yourself

• Set global variables in

  global_var.py

  , especially LOG_DIR where training logs will be stored.
• Set config variables like gender and garment class in

  trainer/base_trainer.py

  (or pass them via command line) and run

  python trainer/base_trainer.py

  to train TailorNet MLP baseline.
• Similarly, run

  python trainer/lf_trainer.py

  to train low frequency predictor and

  trainer/ss2g_trainer.py

  to train shape-style-to-garment(in canonical pose) model.
• Run

  python trainer/hf_trainer.py --shape_style _ _ ...

  to train pivot high frequency predictors for pivots

  _

  ,

  _

  , and so on. See

  DATA_DIR/_/pivots.txt

  to know available pivots.
• Use

  models.tailornet_model.TailorNetModel

  with appropriate logdir arguments to do prediction.

Inference Time

In the paper, we report inference time to be 1-2 ms per frame(depending upon garment) which is averaged inference time over the batch of 21 samples(20-40 ms per batch). Apologies for the ambiguity. Running each sample separately takes almost same time as batch - around 20 ms per frame for all garments. However, note that TailorNet has 21 independent MLPs, so we believe that faster inference time is possible if MLPs are configured to run in parallel on GPU cores.

Misc

• See ./models/skirt_model.md for the explanation of skirt garment model.
• Thanks to Bharat for many fruitful discussions and for

  smpl_lib

  library taken from his MultiGarmentNet repo's lib folder.
• Thanks to Garvita for helping out during the onerous procedure of data generation.

For any doubt or concert about the code, raise an issue on this repository.