Need help with pysot-toolkit?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

StrangerZhang
246 Stars 52 Forks MIT License 11 Commits 17 Opened issues

Description

Python Single Object Tracking Evaluation

Services available

!
?

Need anything else?

Contributors list

No Data

pysot-toolkit

The purpose of this repo is to provide evaluation API of Current Single Object Tracking Dataset, including

Install

git clone https://github.com/StrangerZhang/pysot-toolkit
pip install -r requirements.txt
cd pysot/utils/
python setup.py build_ext --inplace
# if you need to draw graph, you need latex installed on your system

Download Dataset

Download json files used in our toolkit baidu pan or Google Drive

  1. Put CVRP13.json, OTB100.json, OTB50.json in OTB100 dataset directory (you need to copy Jogging to Jogging-1 and Jogging-2, and copy Skating2 to Skating2-1 and Skating2-2 or using softlink)

The directory should have the below format

| -- OTB100/

​ | -- Basketball

​ | ......

​ | -- Woman

​ | -- OTB100.json

​ | -- OTB50.json

​ | -- CVPR13.json

  1. Put all other jsons in the dataset directory like in step 1

Usage

1. Evaluation on VOT2018(VOT2016)

cd /path/to/pysot-toolkit
python bin/eval.py \
    --dataset_dir /path/to/dataset/root \       # dataset path
    --dataset VOT2018 \             # dataset name(VOT2018, VOT2016)
    --tracker_result_dir /path/to/tracker/dir \ # tracker dir
    --trackers ECO UPDT SiamRPNpp           # tracker names 

you will see


|Tracker Name| Accuracy | Robustness | Lost Number | EAO |

| SiamRPNpp | 0.600 | 0.234 | 50.0 | 0.415 | | UPDT | 0.536 | 0.184 | 39.2 | 0.378 | | ECO | 0.484 | 0.276 | 59.0 | 0.280 |


2. Evaluation on OTB100(UAV123, NFS, LaSOT)

converted *.txt tracking results will be released soon

cd /path/to/pysot-toolkit
python bin/eval.py \
    --dataset_dir /path/to/dataset/root \       # dataset path
    --dataset OTB100 \              # dataset name(OTB100, UAV123, NFS, LaSOT)
    --tracker_result_dir /path/to/tracker/dir \ # tracker dir
    --trackers SiamRPN++ C-COT DaSiamRPN ECO  \ # tracker names 
    --num 4 \                   # evaluation thread
    --show_video_level \                # wether to show video results
    --vis                       # draw graph

you will see (Normalized Precision not used in OTB evaluation)


|Tracker name| Success | Norm Precision | Precision |

| SiamRPN++ | 0.696 | 0.000 | 0.914 | | ECO | 0.691 | 0.000 | 0.910 | | C-COT | 0.671 | 0.000 | 0.898 | | DaSiamRPN | 0.658 | 0.000 | 0.880 |



| Tracker name | SiamRPN++ | DaSiamRPN | ECO |

| Video name | success | precision | success | precision | success | precision |

| Basketball | 0.423 | 0.555 | 0.677 | 0.865 | 0.653 | 0.800 | | Biker | 0.728 | 0.932 | 0.319 | 0.448 | 0.506 | 0.832 | | Bird1 | 0.207 | 0.360 | 0.274 | 0.508 | 0.192 | 0.302 | | Bird2 | 0.629 | 0.742 | 0.604 | 0.697 | 0.775 | 0.882 | | BlurBody | 0.823 | 0.879 | 0.759 | 0.767 | 0.713 | 0.894 | | BlurCar1 | 0.803 | 0.917 | 0.837 | 0.895 | 0.851 | 0.934 | | BlurCar2 | 0.864 | 0.926 | 0.794 | 0.872 | 0.883 | 0.931 | ...... | Vase | 0.564 | 0.698 | 0.554 | 0.742 | 0.544 | 0.752 | | Walking | 0.761 | 0.956 | 0.745 | 0.932 | 0.709 | 0.955 | | Walking2 | 0.362 | 0.476 | 0.263 | 0.371 | 0.793 | 0.941 | | Woman | 0.615 | 0.908 | 0.648 | 0.887 | 0.771 | 0.936 |


| OTB100 Success Plot | OTB100 Precision Plot | | --------------------------------- | ----------------------------- | | | |

3. Evaluation on VOT2018-LT

cd /path/to/pysot-toolkit
python bin/eval.py \
    --dataset_dir /path/to/dataset/root \       # dataset path
    --dataset VOT2018-LT \              # dataset name
    --tracker_result_dir /path/to/tracker/dir \ # tracker dir
    --trackers SiamRPN++ MBMD DaSiam-LT \       # tracker names 
    --num 4 \                   # evaluation thread
    --vis \                     # wether to draw graph

you will see


|Tracker Name| Precision | Recall | F1 |

| SiamRPN++ | 0.649 | 0.610 | 0.629 | | MBMD | 0.634 | 0.588 | 0.610 | | DaSiam-LT | 0.627 | 0.588 | 0.607 | | MMLT | 0.574 | 0.521 | 0.546 | | FuCoLoT | 0.538 | 0.432 | 0.479 | | SiamVGG | 0.552 | 0.393 | 0.459 | | SiamFC | 0.600 | 0.334 | 0.429 |


Get Tracking Results of Your Own Tracker

Add pysot-toolkit to your PYTHONPATH

export PYTHONPATH=/path/to/pysot-toolkit:$PYTHONPATH

1. OPE (One Pass Evaluation)

from pysot.datasets import DatasetFactory

dataset = DatasetFactory.create_dataset(name=dataset_name, dataset_root=datset_root, load_img=False) for video in dataset: for idx, (img, gt_bbox) in enumerate(video): if idx == 0: # init your tracker here else: # get tracking result here

2. Restarted Evaluation

from pysot.datasets import DatasetFactory
from pysot.utils.region import vot_overlap

dataset = DatasetFactory.create_dataset(name=dataset_name, dataset_root=datset_root, load_img=False) frame_counter = 0 pred_bboxes = [] for video in dataset: for idx, (img, gt_bbox) in enumerate(video): if idx == frame_counter: # init your tracker here pred_bbox.append(1) elif idx > frame_counter: # get tracking result here pred_bbox = overlap = vot_overlap(pred_bbox, gt_bbox, (img.shape[1], img.shape[0])) if overlap > 0: # continue tracking pred_bboxes.append(pred_bbox) else: # lost target, restart pred_bboxes.append(2) frame_counter = idx + 5 else: pred_bboxes.append(0)

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.