FashionAI_KeyPoint_Detection_Challenge_Keras

by yuanyuanli85

Code for TianChi 2018 FashionAI Cloth KeyPoint Detection Challenge

144 Stars 48 Forks Last release: Not found MIT License 13 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

AiFashion

  • Author: VictorLi, [email protected]
  • Code for FashionAI Global Challenge—Key Points Detection of Apparel 2018 TianChi
  • Rank 45/2322 at 1st round competition, score 0.61
  • Rank 46 at 2nd round competition, score 0.477

Images with detected keypoints

Dress

Dress

Blouse

Blouse

Outwear

Outwear

Skirt

Skirt

Trousers

Trousers

Basic idea

  • The key idea comes from paper Cascaded Pyramid Network for Multi-Person Pose Estimation. We have a 2 stage network called global net and refine net who are U-net like. The network was trained to detect the heatmap of cloth's key points. The backbone network used here is resnet101.
  • To overcome the negative impact from different category,
    input_mask
    was introduced to zero the invalid keypoints. For example, skirt has 4 valid keypoints:
    waistband_left
    ,
    waistband_right
    ,
    hemline_left
    and
    hemline_right
    . In
    input_mask
    , only those valid masks are 1.0 , while other 20 masks are set as zero.
  • On line hard negative mining, at last stage of refinenet, only take the top losses as consideration and ignore the easy part (small loss)

Dependency

  • Keras2.0
  • Tensorflow
  • Opencv/Numpy/Pandas
  • Pretrained model weights, resenet101

Folder Structure

  • data
    : folder to store training and testing images and annotations
  • trained_models
    : folder to store trained models and logs
  • submission
    : folder to store generated submission for evaluation.
  • src
    : folder to put all of source code.
    src/data_gen
    : code for data generator including data augmentation and pre-process
    src/eval
    : code for evaluation, including inference and post-processing.
    src/unet
    : code for cnn model definition, including train, fine-tune, loss, optimizer definition.
    src/top
    :top level code for train, test and demo.

How to train network

  • Download dataset from competition webpage and put it under data.
    data/train
    : data used as train.
    data/test
    : data used for test
  • Download resnet101 model and save it as
    data/resnet101_weights_tf.h5
    .
    Note: all the models here use channel_last dim order.
  • Train all-in-one network from scratch
    python train.py --category all --epochs 30 --network v11 --batchSize 3 --gpuID 2
    
  • The trained model and log will be put under
    trained_models/all/xxxx
    , i.e
    trained_models/all/2018_05_23_15_18_07/

  • The evaluation will run for each epoch and details saved to
    val.log
  • Resume training from a specific model.
    python train.py --gpuID 2 --category all --epochs 30 --network v11 --batchSize 3 --resume True --resumeModel /path/to/model/start/with --initEpoch 6
    

How to test and generate submission

  • Run test and generate submission Below command search the best score from
    modelpath
    and use that to generate submission
    python test.py --gpuID 2 --modelpath ../../trained_models/all/xxx --outpath ../../submission/2018_04_19/ --augment True
    
    The submission will be saved as
    submission.csv

How to run demo

  • Download the pre trained weights from BaiduDisk (password
    1ae2
    ) or GoogleDrive
  • Save it somewhere, i.e
    trained_models/all/fashion_ai_keypoint_weights_epoch28.hdf5
  • Or use your own trained model.
  • Run demo and the cloth with keypoints marked will be displayed.
    python demo.py --gpuID 2 --modelfile ../../trained_models/all/fashion_ai_keypoint_weights_epoch28.hdf5
    

Reference

  • Resnet 101 Keras : https://github.com/statech/resnet

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.