Object Tracking Evaluation

The object tracking benchmark consists of 60 training sequences and 40 testing sequences. Only the vehicles outside the ignoring region are annotated and evaluated in our benchmark. For the benchmark annotation, a group of volunteers manually annotate the 2D bounding boxes in every 5 frames. Annotations in other frames are produced using linear interpolation. All bounding boxes are manually verified, and inaccurate bounding boxes are manually adjusted. In summary, we annotated 8250 vehicles in the benchmark, including 5936 vehicles (i.e., "car": 5177, "bus": 106, "van": 610, "others": 43) in the DETRAC-train set and 2314 vehicles (i.e., "car": 1961, "bus": 199, "van": 123, "others": 31) in the DETRAC-test set. We propose a new evaluation protocol for object tracking and MOT system evaluation. The proposed DETRAC evaluation protocol considers object detection and object tracking in tandem. Specifically, we advocate to evaluate MOT performance using the joint precision vs. recall vs. multi-object tracking performance curves (refer to arXiv paper). Out development kit provides details about the data format as well as utility functions for reading and writing the annotation files.


This demo video shows the ground truth annotations of multi-object tracking in the DETRAC datasets. Colors of the bounding boxes and trajectories reflect different vehicles. Black opaque regions are ignored in the benchmark as general backgrounds. The weather condition, camera status and the vehicle density are presented in the bottom left corner of each frame.

The demo video may take a few seconds to load.

Evaluation Protocol

The DETRAC MOT metrics considers both object detection and object tracking. We take the PR-MOTA curve as an example to explain our novelty. The PR-MOTA curve (see left figure below) is a 3D curve characterizing the relation between object detection performance (precision and recall) and object tracking performance (MOTA). From the PR-MOTA curve, we calculate the integral score Ω to measure multi-object tracking performance, which is the line integral along the PR curve. The scores of other seven metrics, e.g., PR-MOTP, PR-IDS, PR-MT, PR-FRAG, PR-ML, PR-FN, and PR-FP, are similarly calculated. We rank different object tracking methods and MOT systems mainly based on the PR-MOTA scores in evaluation, i.e., larger PR-MOTA score indicates better MOT performance. Other metrics are used for reference only. The hit/miss threshold of the bounding box overlap between an output trajectory and the ground truth is set to be 0.7.


The performance of different MOT systems (four object detection and six object tracking methods) and object tracking methods (the average scores over four detection methods) in DETRAC dataset are presented in the result page.


  • Dataset:
  • Detections:
  • Annotations:
  • Evaluation Software: DETRAC-toolkit (Windows beta)
  • Baseline methods:
    • CEM: A. Andriyenko and K. Schindler. Multi-target tracking by continuous energy minimization. In CVPR, pages 1265-1272, 2011.
    • CMOT: S. H. Bae and K. Yoon. Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In CVPR, pages 1218-1225, 2014.
    • DCT: A. Andriyenko, K. Schindler, and S. Roth. Discrete-continuous optimization for multi-target tracking. In CVPR, pages 1926-1933, 2012.
    • H2T: L. Wen, W. Li, Z. Lei, D. Yi, and S. Z. Li. Multiple target tracking based on undirected hierarchical relation hypergraph. In CVPR, pages 3457-3464, 2014.
    • GOG: H. Pirsiavash, D. Ramanan, and C. C. Fowlkes. Globally-optimal greedy algorithms for tracking a variable number of objects. In CVPR, pages 1201-1208, 2011.
    • IHTLS: C. Dicle, O. I. Camps, and M. Sznaier. The way they move: Tracking multiple targets with similar appearance. In ICCV, pages 2304-2311, 2013.

Related Datasets

  • CAVIAR: A number of video clips recorded acting out the different scenarios
  • KITTI Tracking Benchmark: Multi-object tracking dataset, taken from a moving vehicle with the viewpoint of the tracker
  • MOTChallenge: Multi-pedestrian tracking dataset captured from surveillance cameras
  • TUD Datasets: Includes "TUD Multiview Pedestrians" and "TUD Stadtmitte" Sequences
  • PETS2009: A large crowd dataset focusing on multi-pedestrian tracking and counting
  • EPFL Multi-camera Dataset: Multi-camera multi-pedestrian tracking videos
  • ETHZ Sequences: Inner city sequences captured by mobile platforms
  • PSU-HUB Sequences: Multi-pedetrian tracking videos captured in the PSU student union building


If you use this dataset in your research, please include the following citation in any published results.

  author    = {Longyin Wen and Dawei Du and Zhaowei Cai and Zhen Lei and Ming{-}Ching Chang and
               Honggang Qi and Jongwoo Lim and Ming{-}Hsuan Yang and Siwei Lyu},
  title     = { {DETRAC:} {A} New Benchmark and Protocol for Multi-Object Detection and Tracking},
  journal   = {arXiv CoRR},
  volume    = {abs/1511.04136},
  year      = {2015}