Need help with RelationNet2?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

microsoft
190 Stars 18 Forks MIT License 12 Commits 7 Opened issues

Description

RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder

Services available

!
?

Need anything else?

Contributors list

RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder

by Chi, Cheng and Wei, Fangyun and Hu, Han

Introduction

Existing object detection frameworks are usually built on a single format of objject/part representation, i.e., anchor/proposal rectangle boxes in RetinaNet and Faster R-CNN, center points in FCOS and RepPoints, and corner points in CornerNet. While these different representations usually drive the frameworks to perform well in different aspects, e.g., better classification or finer localization, it is in general difficult to combine these representations in a single framework to make good use of each strength, due to the heterogeneous or non-grid feature extraction by different representations. This paper presents an attention-based decoder module similar as that in Transformer to bridge other representations into a typical object detector built on a single representation format, in an end-to-end fashion. The other representations act as a set of key instances to strengthen the main query representation features in the vanilla detectors. Novel techniques are proposed towards efficient computation of the decoder module, including a key sampling approach and a shared location embedding approach. The proposed module is named bridging visual representations (BVR).

Main Results:

|Model|MS Train|MS Test|mAP|AP50|AP75|Link| |----|-------|-----|----|---|---|---| |retinanetbvrr50|N|N|0.385 |0.591 |0.409|Google| |retinanetbvrx101_dcn|Y|N|0.465| 0.663| 0.506|Google| |fcosbvrx101_dcn|Y|N|0.487| 0.680| 0.529|Google| |atssbvrx101_dcn|Y|N|0.506 |0.695| 0.553|Google|

How to use it

  • ### Install it
    shell
    bash install.sh ${your_code_dir}
    cd ${your_code_dir}
    mkdir -p data 
    ln -s ${your_coco_path} data/coco
    
    where
    your_code_dir
    is your code path and
    your_coco_path
    is the location of extracted coco dataset on your server. For more information, you may refer to getting started
  • ### For testing
    shell
    bash tools/dist_test.sh ${selected_config} 8
    
    where
    selected_config
    is one of provided script under the
    config/bvr
    folder.
  • ### For training
    shell
    bash tools/dist_train.sh ${selected_config} 8
    
    where
    selected_config
    is one of provided script under the
    config/bvr
    folder.
  • ### For more dataset We have not trained or tested on other dataset. If you would like to use it on other data, please refer to mmdetection. ## Citing RelationNet++
@inproceedings{relationnetplusplus2020,
  title={RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder},
  author={Chi, Cheng and Wei, Fangyun and Hu, Han},
  booktitle={NeurIPS},
  year={2020}
}

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.