Need help with deformable-kernels?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

hangg7
185 Stars 27 Forks MIT License 2 Commits 4 Opened issues

Description

Deforming kernels to adapt towards object deformation. In ICLR 2020.

Services available

!
?

Need anything else?

Contributors list

No Data

Deformable Kernels [ICLR 2020] [Website]

Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
Hang Gao*, Xizhou Zhu*, Steve Lin, Jifeng Dai.
In ICLR, 2020.

This repository contains official implementation of deformable kernels.

Table of contents
1. Customized operators for deformable kernels, along with its variants.
2. Instructions to use our operators.
3. Results on ImageNet & COCO benchmarks, with pretrained models for reproduction.
5. Training and evaluation code.

(0) Getting started

PyTorch

  • Get CUDA 10.1 installed on your machine.
  • Install PyTorch (pytorch.org).
  • conda env create -f environment.yml
    .

Apex

  • Install Apex from its official repo. This will require CUDA 10.1 to work with the latest pytorch version (which is
    pytorch=1.3.1
    as being tested against). It is used for fast mix-precision inference and should work out of the box.

Compile our operators

# assume at project root
(
cd deformable_kernels/ops/deform_kernel;
pip install -e .;
)

(1) Customized operators

This repo includes all deformable kernel variants described in our paper, namely:

  • Global Deformable Kernels;
  • Local Deformable Kernels;
  • Local Deformable Kernels integrating with Deformable Convolutions;

Instead of learning offsets on image space, we propose to deform and resample on kernel space. This enables powerful dynamic inference capacity. For more technical details, please refer to their definitions.

We also provide implementations on our rivalries, namely:

Please refer to their module definitions under

deformable_kernels/modules
folder.

(2) Quickstart

The following snippet constructs the deformable kernels we used for our experiments

from deformable_kernels.modules import (
    GlobalDeformKernel2d,
    DeformKernel2d,
    DeformKernelConv2d,
)

global DK with scope size 2, kernel size 1, stride 1, padding 0, depthwise convolution.

gdk = GlobalDeformKernel2d((2, 2), [inplanes], [inplanes], groups=[inplanes])

(local) DK with scope size 4, kernel size 3, stride 1, padding 1, depthwise convolution.

dk = DeformKernel2d((4, 4), [inplanes], [inplanes], 3, 1, 1, groups=[inplanes])

(local) DK integrating with dcn, with kernel & image offsets separately learnt.

dkc = DeformKernelConv2d((4, 4), [inplanes], [inplanes], 3, 1, 1, groups=[inplanes]).

Note that all of our customized operators only support depthwise convolutions now, mainly because that efficiently resampling kernels at runtime is extremely slow if we orthogonally compute over each channel. We are trying to loose this requirement by iterating our CUDA implementation. Any contribuitions are welcome!

(3) Results & pretrained models

Under construction.

(4) Training & evaluation code

Under construction.

(A) License

This project is released under the MIT license.

(B) Citation & Contact

If you find this repo useful for your research, please consider citing this bibtex:

@article{gao2019deformable,
  title={Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation},
  author={Gao, Hang and Zhu, Xizhou and Lin, Steve and Dai, Jifeng},
  journal={arXiv preprint arXiv:1910.02940},
  year={2019}
}

Please contact Hang Gao

 and Xizhou Zhu
 with any comments or feedback.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.