Need help with CPNDet?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Duankaiwen
144 Stars 11 Forks MIT License 22 Commits 6 Opened issues

Description

Corner Proposal Network for Anchor-free, Two-stage Object Detection

Services available

!
?

Need anything else?

Contributors list

# 27,436
MATLAB
Lua
one-sta...
object-...
22 commits

Attention!I failed to open source the trained model! You may need to train by yourselves.

Corner Proposal Network for Anchor-free, Two-stage Object Detection

by Kaiwen Duan, Lingxi Xie, Honggang Qi, Song Bai, Qingming Huang and Qi Tian

The code to train and evaluate the proposed CPN is available here. For more technical details, please refer to our arXiv paper.

We thank Princeton Vision & Learning Lab for providing the original implementation of CornerNet. We also refer to some codes from mmdetection and Objects as Points, we thank them for providing their implementations.

CPN is an anchor-free, two-stage detector which gets trained from scratch. On the MS-COCO dataset, CPN achieves an AP of 49.2%, which is competitive among state-of-the-art object detection methods. In the scenarios that require faster inference speed, CPN can be further accelerated by properly replacing with a lighter backbone (e.g., DLA-34) and not using flip augmentation at the inference stage. In this configuration, CPN reports a 41.6 AP at 26.2 FPS (full test) and a 39.7 AP at 43.3 FPS.

Abstract

The goal of object detection is to determine the class and location of objects in an image. This paper proposes a novel anchor-free, two-stage framework which first extracts a number of object proposals by finding potential corner keypoint combinations and then assigns a class label to each proposal by a standalone classification stage. We demonstrate that these two stages are effective solutions for improving recall and precision, respectively, and they can be integrated into an end-to-end network. Our approach, dubbed Corner Proposal Network (CPN) enjoys the ability to detect objects of various scales and also avoids being confused by a large number of false-positive proposals. On the MS-COCO dataset, CPN achieves an AP of 49.2% which is competitive among state-of-the-art object detection methods. CPN can also fit scenarios that desire for network efficiency. Equipping with a lighter backbone and switching off image flip in inference, CPN achieves 41.6% at 26.2 FPS or 39.7% at 43.3 FPS, surpassing most competitors with the same inference speed.

Introduction

CPN is a framework for object detection with deep convolutional neural networks. You can use the code to train and evaluate a network for object detection on the MS-COCO dataset.

  • It achieves state-of-the-art performance (an AP of 49.2%) on one of the most challenging dataset: MS-COCO.
  • It achieves a good trade-off between accuracy and speed (41.6AP/26.2FPS or 39.7AP/43.3FPS).
  • At the training stage, we use 8 NVIDIA Tesla-V100 (32GB) GPUs on HUAWEI CLOUD to train the network, the traing time is about 9 days, 5 days and 3 days for HG104, HG52 and DLA34, respectively.
  • Our code is written in Pytorch (the master branch works with PyTorch 1.1.0), based on CornerNet, mmdetection, Objects as Points and CenterNet.

If you encounter any problems in using our code, please contact Kaiwen Duan: [email protected]

Architecture

Network_Structure

AP(%) on COCO test-dev and Models

| Backbone | Input Size | AP | AP50 | AP75 | APS | APM | APL | | :-------------------------------------------------------: | :----------------------------------------------------------: | :--: | :-------------: | :-------------: | :------------: | :------------: | :------------: | | DLA34 | ori. | 41.7 | 58.9 | 44.9 | 20.2 | 44.1 | 56.4 | | DLA34 | | 44.5 | 62.3 | 48.3 | 25.2 | 46.7 | 58.2 | | | HG52 | ori. | 43.9 | 61.6 | 47.5 | 23.9 | 46.3 | 57.1 | | HG52 | | 45.8 | 63.9 | 49.7 | 26.8 | 48.4 | 59.4 | | | HG104 | ori. | 47.0 | 65.0 | 51.0 | 26.5 | 50.2 | 60.7 | | HG104 | | 49.2 | 67.3 | 53.7 | 31.0 | 51.9 | 62.4 | |

Notes:

  • denotse multi-scale testing.

Comparison of AR(%) on COCO validation set

| Method | Backbone | AR | AR1+ | AR2+ | AR3+ | AR4+ | AR5:1 | AR6:1 | AR7:1 | AR8:1 | | :----------: | :---------: | :--: | :-------------: | :-------------: | :-------------: | :-------------: | :--------------: | :--------------: | :--------------: | :--------------: | | Faster R-CNN | X-101-64x4d | 57.6 | 73.8 | 77.5 | 79.2 | 86.2 | 43.8 | 43.0 | 34.3 | 23.2 | | FCOS | X-101-64x4d | 64.9 | 82.3 | 87.9 | 89.8 | 95.0 | 45.5 | 40.8 | 34.1 | 23.4 | | CornerNet | HG-104 | 66.8 | 85.8 | 92.6 | 95.5 | 98.5 | 50.1 | 48.3 | 40.4 | 36.5 | | CenterNet | HG-104 | 66.8 | 87.1 | 93.2 | 95.2 | 96.9 | 50.7 | 45.6 | 40.1 | 32.3 | | CPN | HG-104 | 68.8 | 88.2 | 93.7 | 95.8 | 99.1 | 54.4 | 50.6 | 46.2 | 35.4 |

Notes:

  • Here, the average recall is recorded for targets of different aspect ratios and different sizes. To explore the limit of the average recall for each method, we exclude the impacts of bounding-box categories and sorts on recall, and compute it by allowing at most 1000 object proposals. AR1+, AR2+, AR3+ and AR4+ denote box area in the ranges of (962, 2002], (2002, 3002], (3002, 4002] and (4002, ), respectively. 'X' and 'HG' stand for ResNeXt and Hourglass, respectively.

Inference speed COCO validation set

| Backbone | Input Size | Flip | AP | FPS | | :------: | :--------: | :--: | :--: | :--: | | HG52 | ori. | Yes | 43.8 | 9.9 | | HG52 | 0.7x ori. | No | 37.7 | 24.0 | | HG104 | ori. | Yes | 46.8 | 7.3 | | HG104 | 0.7x ori. | No | 40.5 | 17.9 | | DLA34 | ori. | Yes | 41.6 | 26.2 | | DLA34 | ori. | No | 39.7 | 43.3 |

Notes:

  • The FPS is measured on an NVIDIA Tesla-V100 GPU on HUAWEI CLOUD.

Preparation

Please first install Anaconda and create an Anaconda environment using the provided package list.

conda create --name CPN --file conda_packagelist.txt

After you create the environment, activate it.

source activate CPN

Installing some APIs

cd code

and

python setup.py

Downloading MS COCO Data

  • Download the training/validation split we use in our paper from here (originally from Faster R-CNN)
  • Unzip the file and place
    annotations
    under
    /coco
  • Download the images (2014 Train, 2014 Val, 2017 Test) from here
  • Create 3 directories,
    trainval2014
    ,
    minival2014
    and
    testdev2017
    , under
    /coco/images/
  • Copy the training/validation/testing images to the corresponding directories according to the annotation files

Training and Evaluation

To train CPN104 or CPN52 or CPN_DLA34:

python train.py --cfg_file HG104

or

python train.py --cfg_file HG52

or

python train.py --cfg_file DLA34

We provide the configuration file

config/HG104.json
,
config/HG52.json
and
config/DLA34.json
in this repo. If you want to train you own CPN, please adjust the batch size in corresponding onfiguration files to accommodate the number of GPUs that are available to you. Note that if you want train DLA34, you need to firstly download the pre-trained model, and put it under
CPN/cache/nnet/DLA34/pretrain
.

To use the trained model:

python test.py --cfg_file HG104 --testiter 220000 --split 

or

python test.py --cfg_file HG52 --testiter 270000 --split 

or

python test.py --cfg_file DLA34 --testiter 270000 --split 

where

 = validation or testing
.

You need to download the corresponding models and put them under

CPN/cache/nnet
.

You can add

--no_flip
for testing without flip augmentation.

You can also add

--debug
to visualize some detection results (uncomment the codes from line 152 to 182 in
CPN/code/test/coco.py
).

We also include a configuration file for multi-scale evaluation, which is

HG104-multi_scale.json
and
HG52-multi_scale.json
and
DLA34-multi_scale.json
in this repo, respectively.

To use the multi-scale configuration file:

python test.py --cfg_file HG104 --testiter  --split  --suffix multi_scale

or

python test.py --cfg_file HG52 --testiter  --split  --suffix multi_scale

or

python test.py --cfg_file DLA34 --testiter  --split  --suffix multi_scale

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.