The object tracking benchmark consists of 60 training sequences and 40 testing sequences. Only the vehicles outside the ignoring region are annotated and evaluated in our benchmark. For the benchmark annotation, a group of volunteers manually annotate the 2D bounding boxes in every 5 frames. Annotations in other frames are produced using linear interpolation. All bounding boxes are manually verified, and inaccurate bounding boxes are manually adjusted. In summary, we annotated 8250 vehicles in the benchmark, including 5936 vehicles (i.e., "car": 5177, "bus": 106, "van": 610, "others": 43) in the DETRAC-train set and 2314 vehicles (i.e., "car": 1961, "bus": 199, "van": 123, "others": 31) in the DETRAC-test set. We propose a new evaluation protocol for object tracking and MOT system evaluation. The proposed DETRAC evaluation protocol considers object detection and object tracking in tandem. Specifically, we advocate to evaluate MOT performance using the joint precision vs. recall vs. multi-object tracking performance curves (refer to arXiv paper). Out development kit provides details about the data format as well as utility functions for reading and writing the annotation files.

This demo video shows the ground truth annotations of multi-object tracking in the DETRAC datasets. Colors of the bounding boxes and trajectories reflect different vehicles. Black opaque regions are ignored in the benchmark as general backgrounds. The weather condition, camera status and the vehicle density are presented in the bottom left corner of each frame.

The demo video may take a few seconds to load.

The DETRAC MOT metrics considers both object detection and object tracking. We take the PR-MOTA curve as an example to explain our novelty. The PR-MOTA curve (see left figure below) is a 3D curve characterizing the relation between object detection performance (precision and recall) and object tracking performance (MOTA). From the PR-MOTA curve, we calculate the integral score Ω^∗ to measure multi-object tracking performance, which is the line integral along the PR curve. The scores of other seven metrics, e.g., PR-MOTP, PR-IDS, PR-MT, PR-FRAG, PR-ML, PR-FN, and PR-FP, are similarly calculated. We rank different object tracking methods and MOT systems mainly based on the PR-MOTA scores in evaluation, i.e., larger PR-MOTA score indicates better MOT performance. Other metrics are used for reference only. The hit/miss threshold of the bounding box overlap between an output trajectory and the ground truth is set to be 0.7.

The performance of different MOT systems (four object detection and six object tracking methods) and object tracking methods (the average scores over four detection methods) in DETRAC dataset are presented in the result page.