Here is a pytorch implementation of deeplabv3+ supporting ResNet(79.155%) and Xception(79.945%). Multi-scale & flip test and COCO dataset interface has been finished.
Most recurrent update: 2021.01.08 - Newest version codebase has released here, which releases output_stride=8 deeplabv3+ model.
2019.01.21 - Updata the code for paper performance achieved! Now deeplabv3+res101 achieve 79.155% and deeplabv3+xception achieve 79.945% on PASCAL VOC 2012 val set. The main bug is the missing of
patch_replication_callback()function of Synchronized Batch Normalization.
2018.11.26 - Update including support Xception network, multi-scale test, network output stride modification, pure train set finetuning, and more dataset interface (PASCAL Context, Cityscapes, ADE20K)
2018.09.28 - Add python evaluation func in
2018.09.21 - Fix some bugs in
Here is a pytorch implementation of deeplabv3+. The project support variants of dataset including MS COCO object detection dataset, PASCAL VOC, PASCAL Context, Cityscapes, ADE20K.
Ubuntu16.04 + python3.6 + pytorch0.4 + CUDA8.0
You should run the code on more than 2 GPU because only multi-GPU version is supported at present.
Anaconda virtual environment is suggested.
Please install tensorboardX for loss curves and segmentation visualization.
SynchronizedBatchNorm (from vacancy) has been used in our model, which will not be installed manually.
Download VOC2012 dataset and VOCdevkit, keep the VOC2012 folder in VOCdevkit as
$root/data/VOCdevkit/VOC2012/...., or you can modified path in
Download VOC augmented segmentation dataset from DrSleep. Put the annotations in
$root/data/VOCdevkit/VOC2012/SegmentationClass/and name list file(
Download COCO2017 object detection dataset here for pretrain, keep the path as
$root/data/MSCOCO/...., or change it in
(optional) Install Matlab for VOC2012 evaluation (VOC official evaluation code in VOCdevkit is written by Matlab, and we have reimplement a python version)
Now the project support modified ResNet and Xception network as backbone.
For ResNet backbone, the structure change the dilation of layer4 as [2,2,2].
As for Xception, we keep the same structure with offical tensorflow code and transform the pretrained parameter file from
Now ResNet101 and Xception achieve paper performance without multi-scale and flip test and finetuning on VOC2012aug dataset. The models with different setting will be released once paper performance is achieved. You can also finetuning the model by yourself where COCO dataset inferface has already be released. Discussion about finetuning tricks is welcomed by email(yude.wang at outlook.com) or issues!!!
Here are some pretrained model (46epoch, about 30k iterations) for download, the performance is evaluated on PASCAL VOC 2012 val set. (deeplabv3+ models are trained on multi GPUs, it may cause error when loading by cpu version):
| backbone | output stride | multi-scale & flip test | paper performance(mIoU) | our performance(mIoU) | |----------|---------------|-------------------------|-------------------------|-----------------------| | xceptionpytorchimagnet | | | | | deeplabv3+res101 | 16 | False | 78.85% | 79.155% | | | 16 | True | 80.22% | 79.916% | | deeplabv3+xception | 16 | False | 79.93% | 79.945% | | | 16 | True | 81.44% | 81.087% |
Model checkpoint with higher performance will be updata once achieve.
Please check the configuration file
$root/experiment/project_name/config.pyfirst to meet your requirements.
Please check the pretrained checkpoint path in
Please set visible gpu devices before training.
Then you can train the network as you wish.
cd $root/experiment/project_name python train.py
You can also test the performance of trained network by:
cd $root/experiment/project_name python test.py
Discussion about learning tricks is welcomed (yude.wang at outlook.com).
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam. In ECCV, 2018.
Rethinking Atrous Convolution for Semantic Image Segmentation Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. arXiv: 1706.05587, 2017.