PyTorch implementation of Grad-CAM, vanilla/guided backpropagation, deconvnet, and occlusion sensitivity maps
PyTorch implementation of Grad-CAM (Gradient-weighted Class Activation Mapping) [1] in image classification. This repository also contains implementations of vanilla backpropagation, guided backpropagation [2], deconvnet [2], and guided Grad-CAM [1], occlusion sensitivity maps [3].
Python 2.7 / 3.+
$ pip install click opencv-python matplotlib tqdm numpy $ pip install "torch>=0.4.1" torchvision
python main.py [DEMO_ID] [OPTIONS]
Demo ID:
Options:
-i,
--image-paths: image path, which can be provided multiple times (required)
-a,
--arch: a model name from
torchvision.models, e.g. "resnet152" (required)
-t,
--target-layer: a module name to be visualized, e.g. "layer4.2" (required)
-k,
--topk: the number of classes to generate (default: 3)
-o,
--output-dir: a directory to store results (default: ./results)
--cuda/--cpu: GPU or CPU
The command above generates, for top k classes:
The guided-* do not support
F.relubut only
nn.ReLUin this codes. For instance, off-the-shelf
inception_v3cannot cut off negative gradients during backward operation (issue #2).
Generate all kinds of visualization maps given a torchvision model, a target layer, and images.
python main.py demo1 -a resnet152 -t layer4 \ -i samples/cat_dog.png -i samples/vegetables.jpg # You can add more images
| Predicted class | #1 boxer | #2 bull mastiff | #3 tiger cat |
| :----------------------------------------: | :---------------------------------------------------: | :----------------------------------------------------------: | :-------------------------------------------------------: |
| Grad-CAM [1] | |
|
|
| Vanilla backpropagation |
|
|
|
| "Deconvnet" [2] |
|
|
|
| Guided backpropagation [2] |
|
|
|
| Guided Grad-CAM [1] |
|
|
|
Grad-CAM with different models for "bull mastiff" class
| Model |
resnet152|
vgg19|
vgg19_bn|
densenet201|
squeezenet1_1| | :--------------------------: | :---------------------------------------------------: | :-------------------------------------------------: | :----------------------------------------------------: | :-------------------------------------------------------: | :---------------------------------------------------------: | | Layer |
layer4|
features|
features|
features|
features| | Grad-CAM [1] |
Generate Grad-CAM maps for "bull mastiff" class, at different layers of ResNet-152 (hardcoded).
python main.py demo2 -i samples/cat_dog.png
| Layer |
relu|
layer1|
layer2|
layer3|
layer4| | :--------------------------: | :-------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | | Grad-CAM [1] |
Generate the occlusion sensitivity map [1, 3] based on logit scores. The red and blue regions indicate a relative increase and decrease from non-occluded scores respectively: the blue regions are critical!
python main.py demo3 -a resnet152 -i samples/cat_dog.png
| Patch size | 10x10 | 15x15 | 25x25 | 35x35 | 45x45 | 90x90 |
| :----------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: | :---------------------------------------------------: |
| "boxer" sensitivity | |
|
|
|
|
|
| "bull mastiff" sensitivity |
|
|
|
|
|
|
| "tiger cat" sensitivity |
|
|
|
|
|
|
This demo takes much time to compute per-pixel logits. You can control the resolution by changing sampling stride (
--stride), or increasing batch size as to fit on your GPUs (
--n-batches). The model is wrapped with
torch.nn.DataParallelso that runs on multiple GPUs by default.