Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training

By Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, Xilin Chen.

This project is based on maskrcnn-benchmark.

[2020.7] Dynamic R-CNN is officially included in MMDetection V2.2, many thanks to @xvjiarui and @hellock for migrating the code.

Abstract

Although two-stage object detectors have continuously advanced the state-of-the-art performance in recent years, the training process itself is far from crystal. In this work, we first point out the inconsistency problem between the fixed network settings and the dynamic training procedure, which greatly affects the performance. For example, the fixed label assignment strategy and regression loss function cannot fit the distribution change of proposals and are harmful to training high quality detectors. Then, we propose Dynamic R-CNN to adjust the label assignment criteria (IoU threshold) and the shape of regression loss function (parameters of SmoothL1 Loss) automatically based on the statistics of proposals during training. This dynamic design makes better use of the training samples and pushes the detector to fit more high quality samples. Specifically, our method improves upon ResNet-50-FPN baseline with 1.9% AP and 5.5% AP90 on the MS COCO dataset with no extra overhead. For more details, please refer to our paper.

Models

1. 1x

   ,

   2x

   and

   3x

   mean the model is trained for 90K, 180K and 270K iterations, respectively.
2. For

   Multi-scale training

   , the shorter side of images is randomly chosen from (400, 600, 800, 1000, 1200), and the longer side is 1400. We also extend the training time by

   1.5x

   under this setting.

3. dcnv2

   denotes deformable convolutional networks v2. We follow the same setting as maskrcnn-benchmark. Note that the result of this version is slightly lower than that of mmdetection.
4. All results in the table are obtained using a single model with no extra testing tricks. Additionally, adopting multi-scale testing on model

   Dynamic_RCNN_r101_dcnv2_fpn_3x

   achieves 49.2% in AP on COCO test-dev. Please set

   TEST.BBOX_AUG.ENABLED = True

   in the

   config.py

   to enable multi-scale testing. Here we use five scales with shorter sides (800, 1000, 1200, 1400, 1600) and the longer side is 2000 pixels. Note that Dynamic R-CNN*(50.1% AP) in Table 9 is implemented using MMDetection v1.1, please refer to this link.
5. If you want to test the model provided by us, please refer to Testing.

Getting started

Installation

0. Requirements

• pytorch (v1.0.1.post2, other version have not been tested)
• torchvision (v0.2.2.post3, other version have not been tested)
• cocoapi
• matplotlib
• tqdm
• cython
• easydict
• opencv

Anaconda3 is recommended here since it integrates many useful packages. Please make sure that your conda is setup properly with the right environment. Then install

pytorch

torchvision

pip install torch==1.0.1.post2
pip install torchvision==0.2.2.post3

Other dependencies will be installed during

setup

1. Clone this repo

git clone https://github.com/hkzhang95/DynamicRCNN.git

2. Compile kernels

Please make sure your

CUDA

PATH

CUDA-9.0

cd ${DynamicRCNN_ROOT}
python setup.py build develop

3. Prepare data and output directory

cd ${DynamicRCNN_ROOT}
mkdir data
mkdir output

Prepare data and pretrained models: - COCO dataset - ImageNet Pretrained Models from Detectron

Then organize them as follows:

DynamicRCNN
├── dynamic_rcnn
├── models
├── output
├── data
│   ├── basemodels/R-50.pkl
│   ├── coco
│   │   ├── annotations
│   │   ├── train2017(2014)
│   │   ├── val2017(2014)

Training

We use

torch.distributed.launch

cd models/zhanghongkai/dynamic_rcnn/coco/dynamic_rcnn_r50_fpn_1x
python -m torch.distributed.launch --nproc_per_node=8 train.py

Outputs

Training and testing logs will be saved automatically in the

output

models

For example, the experiment directory and log directory are formed as follows:

models/zhanghongkai/dynamic_rcnn/coco/dynamic_rcnn_r50_fpn_1x
output/zhanghongkai/dynamic_rcnn/coco/dynamic_rcnn_r50_fpn_1x

And you can link the

log

python config.py -log

Testing

Using

-i

# for regular testing and evaluation
python -m torch.distributed.launch --nproc_per_node=8 test.py
# for specified iteration
python -m torch.distributed.launch --nproc_per_node=8 test.py -i $iteration_number

If you want to test our provided model, just download the model, move it to the corresponding log directory and create a symbolic link like follows:

# example for Dynamic_RCNN_r50_fpn_1x
cd models/zhanghongkai/dynamic_rcnn/coco/dynamic_rcnn_r50_fpn_1x
python config.py -log
realpath log | xargs mkdir
mkdir -p log/checkpoints
mv path/to/the/model log/checkpoints
realpath log/checkpoints/dynamic_rcnn_r50_fpn_1x_test_model_0090000.pth last_checkpoint | xargs ln -s

Then you can follow the regular testing and evaluation process.

Third-party resources

• MxNet implementation: SimpleDet

Acknowledgement

• maskrcnn-benchmark
• lightheadrcnn

Citations

Please consider citing our paper in your publications if it helps your research:

@inproceedings{DynamicRCNN,
    author = {Zhang, Hongkai and Chang, Hong and Ma, Bingpeng and Wang, Naiyan and Chen, Xilin},
    title = {Dynamic {R-CNN}: Towards High Quality Object Detection via Dynamic Training},
    booktitle = {ECCV},
    year = {2020}
}