Github url


by facebookresearch

facebookresearch /maskrcnn-benchmark

Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in P...

7.8K Stars 2.2K Forks Last release: 11 months ago (v0.1) MIT License 230 Commits 1 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Faster R-CNN and Mask R-CNN in PyTorch 1.0

maskrcnn-benchmark has been deprecated. Please see detectron2, which includes implementations for all models in maskrcnn-benchmark

This project aims at providing the necessary building blocks for easily creating detection and segmentation models using PyTorch 1.0.

alt text


  • PyTorch 1.0: RPN, Faster R-CNN and Mask R-CNN implementations that matches or exceeds Detectron accuracies
  • Very fast: up to 2x faster than Detectron and 30% faster than mmdetection during training. See for more details.
  • Memory efficient: uses roughly 500MB less GPU memory than mmdetection during training
  • Multi-GPU training and inference
  • Mixed precision training: trains faster with less GPU memory on NVIDIA tensor cores.
  • Batched inference: can perform inference using multiple images per batch per GPU
  • CPU support for inference: runs on CPU in inference time. See our webcam demo for an example
  • Provides pre-trained models for almost all reference Mask R-CNN and Faster R-CNN configurations with 1x schedule.

Webcam and Jupyter notebook demo

We provide a simple webcam demo that illustrates how you can use


for inference: ```bash cd demo

by default, it runs on the GPU

for best results, use min-image-size 800

python --min-image-size 800

can also run it on the CPU

python --min-image-size 300 MODEL.DEVICE cpu

or change the model that you want to use

python --config-file ../configs/caffe2/e2e_mask_rcnn_R_101_FPN_1x_caffe2.yaml --min-image-size 300 MODEL.DEVICE cpu

in order to see the probability heatmaps, pass --show-mask-heatmaps

python --min-image-size 300 --show-mask-heatmaps MODEL.DEVICE cpu

for the keypoint demo

python --config-file ../configs/caffe2/e2e_keypoint_rcnn_R_50_FPN_1x_caffe2.yaml --min-image-size 300 MODEL.DEVICE cpu ```

A notebook with the demo can be found in demo/Mask_R-CNN_demo.ipynb.


Check for installation instructions.

Model Zoo and Baselines

Pre-trained models, baselines and comparison with Detectron and mmdetection can be found in

Inference in a few lines

We provide a helper class to simplify writing inference pipelines using pre-trained models. Here is how we would do it. Run this from the


folder: ```python from maskrcnn_benchmark.config import cfg from predictor import COCODemo

config_file = "../configs/caffe2/e2e_mask_rcnn_R_50_FPN_1x_caffe2.yaml"

update the config options with the config file


manual override some options

cfg.merge_from_list(["MODEL.DEVICE", "cpu"])

coco_demo = COCODemo( cfg, min_image_size=800, confidence_threshold=0.7, )

load image and then run prediction

image = ... predictions = coco_demo.run_on_opencv_image(image) ```

Perform training on COCO dataset

For the following examples to work, you need to first install



You will also need to download the COCO dataset. We recommend to symlink the path to the coco dataset to


as follows

We use




sets from Detectron

# symlink the coco dataset cd ~/github/maskrcnn-benchmark mkdir -p datasets/coco ln -s /path\_to\_coco\_dataset/annotations datasets/coco/annotations ln -s /path\_to\_coco\_dataset/train2014 datasets/coco/train2014 ln -s /path\_to\_coco\_dataset/test2014 datasets/coco/test2014 ln -s /path\_to\_coco\_dataset/val2014 datasets/coco/val2014 # or use COCO 2017 version ln -s /path\_to\_coco\_dataset/annotations datasets/coco/annotations ln -s /path\_to\_coco\_dataset/train2017 datasets/coco/train2017 ln -s /path\_to\_coco\_dataset/test2017 datasets/coco/test2017 ln -s /path\_to\_coco\_dataset/val2017 datasets/coco/val2017 # for pascal voc dataset: ln -s /path\_to\_VOCdevkit\_dir datasets/voc




  • valminusminival

You can also configure your own paths to the datasets. For that, all you need to do is to modify


to point to the location where your dataset is stored. You can also create a new


file which implements the same two classes, and pass it as a config argument


during training.

Single GPU training

Most of the configuration files that we provide assume that we are running on 8 GPUs. In order to be able to run it on fewer GPUs, there are a few possibilities:

1. Run the following without modifications

python /path\_to\_maskrcnn\_benchmark/tools/train\ --config-file "/path/to/config/file.yaml"

This should work out of the box and is very similar to what we should do for multi-GPU training. But the drawback is that it will use much more GPU memory. The reason is that we set in the configuration files a global batch size that is divided over the number of GPUs. So if we only have a single GPU, this means that the batch size for that GPU will be 8x larger, which might lead to out-of-memory errors.

If you have a lot of memory available, this is the easiest solution.

2. Modify the cfg parameters

If you experience out-of-memory errors, you can reduce the global batch size. But this means that you'll also need to change the learning rate, the number of iterations and the learning rate schedule.

Here is an example for Mask R-CNN R-50 FPN with the 1x schedule:

bash python tools/train\ --config-file "configs/e2e\_mask\_rcnn\_R\_50\_FPN\_1x.yaml" SOLVER.IMS\_PER\_BATCH 2 SOLVER.BASE\_LR 0.0025 SOLVER.MAX\_ITER 720000 SOLVER.STEPS "(480000, 640000)" TEST.IMS\_PER\_BATCH 1 MODEL.RPN.FPN\_POST\_NMS\_TOP\_N\_TRAIN 2000

This follows the scheduling rules from Detectron.Note that we have multiplied the number of iterations by 8x (as well as the learning rate schedules), and we have divided the learning rate by 8x.

We also changed the batch size during testing, but that is generally not necessary because testing requires much less memory than training.

Furthermore, we set


as the proposals are selected for per the batch rather than per image in the default training. The value is calculated by 1000 x images-per-gpu. Here we have 2 images per GPU, therefore we set the number as 1000 x 2 = 2000. If we have 8 images per GPU, the value should be set as 8000. Note that this does not apply if


is set to


during training. See #672 for more details.

Multi-GPU training

We use internally


in order to launch multi-gpu training. This utility function from PyTorch spawns as many Python processes as the number of GPUs we want to use, and each Python process will only use a single GPU.

export NGPUS=8 python -m torch.distributed.launch --nproc\_per\_node=$NGPUS /path\_to\_maskrcnn\_benchmark/tools/train\ --config-file "path/to/config/file.yaml" MODEL.RPN.FPN\_POST\_NMS\_TOP\_N\_TRAIN images\_per\_gpu x 1000

Note we should set


follow the rule in Single-GPU training.

Mixed precision training

We currently use APEX to add Automatic Mixed Precision support. To enable, just do Single-GPU or Multi-GPU training and set

DTYPE "float16"


export NGPUS=8 python -m torch.distributed.launch --nproc\_per\_node=$NGPUS /path\_to\_maskrcnn\_benchmark/tools/train\ --config-file "path/to/config/file.yaml" MODEL.RPN.FPN\_POST\_NMS\_TOP\_N\_TRAIN images\_per\_gpu x 1000 DTYPE "float16"

If you want more verbose logging, set


. See Mixed Precision Training guide for more details.


You can test your model directly on single or multiple gpus. Here is an example for Mask R-CNN R-50 FPN with the 1x schedule on 8 GPUS:

bash export NGPUS=8 python -m torch.distributed.launch --nproc\_per\_node=$NGPUS /path\_to\_maskrcnn\_benchmark/tools/test\ --config-file "configs/e2e\_mask\_rcnn\_R\_50\_FPN\_1x.yaml" TEST.IMS\_PER\_BATCH 16

To calculate mAP for each class, you can simply modify a few lines in See #524 for more details.


For more information on some of the main abstractions in our implementation, see

Adding your own dataset

This implementation adds support for COCO-style datasets. But adding support for training on a new dataset can be done as follows: ```python from maskrcnn_benchmark.structures.bounding_box import BoxList

class MyDataset(object): def init(self, ...): # as you would do normally

def \_\_getitem\_\_(self, idx): # load the image as a PIL Image image = ... # load the bounding boxes as a list of list of boxes # in this case, for illustrative purposes, we use # x1, y1, x2, y2 order. boxes = [[0, 0, 10, 10], [10, 20, 50, 50]] # and labels labels = torch.tensor([10, 20]) # create a BoxList from the boxes boxlist = BoxList(boxes, image.size, mode="xyxy") # add the labels to the boxlist boxlist.add\_field("labels", labels) if self.transforms: image, boxlist = self.transforms(image, boxlist) # return the image, the boxlist and the idx in your dataset return image, boxlist, idx def get\_img\_info(self, idx): # get img\_height and img\_width. This is used if # we want to split the batches according to the aspect ratio # of the image, as it can be more efficient than loading the # image from disk return {"height": img\_height, "width": img\_width}
That's it. You can also add extra fields to the boxlist, such as segmentation masks (using `structures.segmentation_mask.SegmentationMask`), or even your own instance type. For a full example of how the `COCODataset` is implemented, check [`maskrcnn_benchmark/data/datasets/`](maskrcnn\_benchmark/data/datasets/ Once you have created your dataset, it needs to be added in a couple of places: - [`maskrcnn_benchmark/data/datasets/`](maskrcnn\_benchmark/data/datasets/\_\_init\_\ add it to `__all__` - [`maskrcnn_benchmark/config/`](maskrcnn\_benchmark/config/paths\ `DatasetCatalog.DATASETS` and corresponding `if` clause in `DatasetCatalog.get()` ### Testing While the aforementioned example should work for training, we leverage the cocoApi for computing the accuracies during testing. Thus, test datasets should currently follow the cocoApi for now. To enable your dataset for testing, add a corresponding if statement in [`maskrcnn_benchmark/data/datasets/evaluation/`](maskrcnn\_benchmark/data/datasets/evaluation/\_\_init\_\ ```python if isinstance(dataset, datasets.MyDataset): return coco\_evaluation(\*\*args)

Finetuning from Detectron weights on custom datasets

Create a script


like here. You can decide which keys to be removed and which keys to be kept by modifying the script.

Then you can simply point the converted model path in the config file by changing



For further information, please refer to #15.


If you have issues running or compiling this code, we have compiled a list of common issues If your issue is not present there, please feel free to open a new issue.


Please consider citing this project in your publications if it helps your research. The following is a BibTeX reference. The BibTeX entry requires the


LaTeX package.

@misc{massa2018mrcnn, author = {Massa, Francisco and Girshick, Ross}, title = {{maskrcnn-benchmark: Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch}}, year = {2018}, howpublished = {\url{}}, note = {Accessed: [Insert date here]} }

Projects using maskrcnn-benchmark


maskrcnn-benchmark is released under the MIT license. See LICENSE for additional details.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.