Need help with CPNDet?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

Duankaiwen
172 Stars 21 Forks MIT License 22 Commits 10 Opened issues

#### Description

Corner Proposal Network for Anchor-free, Two-stage Object Detection

!
?

# 29,584
MATLAB
Lua
Jupyter...
one-sta...
22 commits

# Corner Proposal Network for Anchor-free, Two-stage Object Detection

The code to train and evaluate the proposed CPN is available here. For more technical details, please refer to our arXiv paper.

We thank Princeton Vision & Learning Lab for providing the original implementation of CornerNet. We also refer to some codes from mmdetection and Objects as Points, we thank them for providing their implementations.

CPN is an anchor-free, two-stage detector which gets trained from scratch. On the MS-COCO dataset, CPN achieves an AP of 49.2%, which is competitive among state-of-the-art object detection methods. In the scenarios that require faster inference speed, CPN can be further accelerated by properly replacing with a lighter backbone (e.g., DLA-34) and not using flip augmentation at the inference stage. In this configuration, CPN reports a 41.6 AP at 26.2 FPS (full test) and a 39.7 AP at 43.3 FPS.

## Abstract

The goal of object detection is to determine the class and location of objects in an image. This paper proposes a novel anchor-free, two-stage framework which first extracts a number of object proposals by finding potential corner keypoint combinations and then assigns a class label to each proposal by a standalone classification stage. We demonstrate that these two stages are effective solutions for improving recall and precision, respectively, and they can be integrated into an end-to-end network. Our approach, dubbed Corner Proposal Network (CPN) enjoys the ability to detect objects of various scales and also avoids being confused by a large number of false-positive proposals. On the MS-COCO dataset, CPN achieves an AP of 49.2% which is competitive among state-of-the-art object detection methods. CPN can also fit scenarios that desire for network efficiency. Equipping with a lighter backbone and switching off image flip in inference, CPN achieves 41.6% at 26.2 FPS or 39.7% at 43.3 FPS, surpassing most competitors with the same inference speed.

## Introduction

CPN is a framework for object detection with deep convolutional neural networks. You can use the code to train and evaluate a network for object detection on the MS-COCO dataset.

• It achieves state-of-the-art performance (an AP of 49.2%) on one of the most challenging dataset: MS-COCO.
• It achieves a good trade-off between accuracy and speed (41.6AP/26.2FPS or 39.7AP/43.3FPS).
• At the training stage, we use 8 NVIDIA Tesla-V100 (32GB) GPUs on HUAWEI CLOUD to train the network, the traing time is about 9 days, 5 days and 3 days for HG104, HG52 and DLA34, respectively.
• Our code is written in Pytorch (the master branch works with PyTorch 1.1.0), based on CornerNet, mmdetection, Objects as Points and CenterNet.

## AP(%) on COCO test-dev and Models

| Backbone | Input Size | AP | AP50 | AP75 | APS | APM | APL | | :-------------------------------------------------------: | :----------------------------------------------------------: | :--: | :-------------: | :-------------: | :------------: | :------------: | :------------: | | DLA34 | ori. | 41.7 | 58.9 | 44.9 | 20.2 | 44.1 | 56.4 | | DLA34 $\ddagger$ | $\leq&space;1.8\times$ | 44.5 | 62.3 | 48.3 | 25.2 | 46.7 | 58.2 | | | HG52 | ori. | 43.9 | 61.6 | 47.5 | 23.9 | 46.3 | 57.1 | | HG52 $\ddagger$ | $\leq&space;1.8\times$ | 45.8 | 63.9 | 49.7 | 26.8 | 48.4 | 59.4 | | | HG104 | ori. | 47.0 | 65.0 | 51.0 | 26.5 | 50.2 | 60.7 | | HG104 $\ddagger$ | $\leq&space;1.8\times$ | 49.2 | 67.3 | 53.7 | 31.0 | 51.9 | 62.4 | |

Notes:

• $\ddagger$ denotse multi-scale testing.

## Comparison of AR(%) on COCO validation set

| Method | Backbone | AR | AR1+ | AR2+ | AR3+ | AR4+ | AR5:1 | AR6:1 | AR7:1 | AR8:1 | | :----------: | :---------: | :--: | :-------------: | :-------------: | :-------------: | :-------------: | :--------------: | :--------------: | :--------------: | :--------------: | | Faster R-CNN | X-101-64x4d | 57.6 | 73.8 | 77.5 | 79.2 | 86.2 | 43.8 | 43.0 | 34.3 | 23.2 | | FCOS | X-101-64x4d | 64.9 | 82.3 | 87.9 | 89.8 | 95.0 | 45.5 | 40.8 | 34.1 | 23.4 | | CornerNet | HG-104 | 66.8 | 85.8 | 92.6 | 95.5 | 98.5 | 50.1 | 48.3 | 40.4 | 36.5 | | CenterNet | HG-104 | 66.8 | 87.1 | 93.2 | 95.2 | 96.9 | 50.7 | 45.6 | 40.1 | 32.3 | | CPN | HG-104 | 68.8 | 88.2 | 93.7 | 95.8 | 99.1 | 54.4 | 50.6 | 46.2 | 35.4 |

Notes:

• Here, the average recall is recorded for targets of different aspect ratios and different sizes. To explore the limit of the average recall for each method, we exclude the impacts of bounding-box categories and sorts on recall, and compute it by allowing at most 1000 object proposals. AR1+, AR2+, AR3+ and AR4+ denote box area in the ranges of (962, 2002], (2002, 3002], (3002, 4002] and (4002, $\infty$), respectively. 'X' and 'HG' stand for ResNeXt and Hourglass, respectively.

## Inference speed COCO validation set

| Backbone | Input Size | Flip | AP | FPS | | :------: | :--------: | :--: | :--: | :--: | | HG52 | ori. | Yes | 43.8 | 9.9 | | HG52 | 0.7x ori. | No | 37.7 | 24.0 | | HG104 | ori. | Yes | 46.8 | 7.3 | | HG104 | 0.7x ori. | No | 40.5 | 17.9 | | DLA34 | ori. | Yes | 41.6 | 26.2 | | DLA34 | ori. | No | 39.7 | 43.3 |

Notes:

• The FPS is measured on an NVIDIA Tesla-V100 GPU on HUAWEI CLOUD.

## Preparation

Please first install Anaconda and create an Anaconda environment using the provided package list.

conda create --name CPN --file conda_packagelist.txt


After you create the environment, activate it.

source activate CPN


## Installing some APIs

cd code


and

python setup.py


• Download the training/validation split we use in our paper from here (originally from Faster R-CNN)
• Unzip the file and place
annotations
under
/coco
• Download the images (2014 Train, 2014 Val, 2017 Test) from here
• Create 3 directories,
trainval2014
,
minival2014
and
testdev2017
, under
/coco/images/
• Copy the training/validation/testing images to the corresponding directories according to the annotation files

## Training and Evaluation

To train CPN104 or CPN52 or CPN_DLA34:

python train.py --cfg_file HG104


or

python train.py --cfg_file HG52


or

python train.py --cfg_file DLA34


We provide the configuration file

config/HG104.json
,
config/HG52.json
and
config/DLA34.json
in this repo. If you want to train you own CPN, please adjust the batch size in corresponding onfiguration files to accommodate the number of GPUs that are available to you. Note that if you want train DLA34, you need to firstly download the pre-trained model, and put it under
CPN/cache/nnet/DLA34/pretrain
.

To use the trained model:

python test.py --cfg_file HG104 --testiter 220000 --split


or

python test.py --cfg_file HG52 --testiter 270000 --split


or

python test.py --cfg_file DLA34 --testiter 270000 --split



where

 = validation or testing
.

CPN/cache/nnet
.

--no_flip
for testing without flip augmentation.

--debug
to visualize some detection results (uncomment the codes from line 152 to 182 in
CPN/code/test/coco.py
).

We also include a configuration file for multi-scale evaluation, which is

HG104-multi_scale.json
and
HG52-multi_scale.json
and
DLA34-multi_scale.json
in this repo, respectively.

To use the multi-scale configuration file:

python test.py --cfg_file HG104 --testiter  --split  --suffix multi_scale



or

python test.py --cfg_file HG52 --testiter  --split  --suffix multi_scale



or

python test.py --cfg_file DLA34 --testiter  --split  --suffix multi_scale