Evaluation Metrics

This table describes the evaluation metrics used in the DETRAC-tracking benchmark.

Measurement Better Perfect Description
PR-MOTA higher 100% Multiple Object Tracking Accuracy along the PR curve. This measure combines three error sources: false positives, missed targets and identity switches.
PR-MOTP higher 100% Multiple Object Tracking Precision along the PR curve. The misalignment between the annotated and the predicted bounding boxes.
PR-MT higher 100% Mostly tracked targets along the PR curve. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span.
PR-ML lower 0% Mostly lost targets along the PR curve. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span.
PR-IDS lower 0 The total number of identity switches along the PR curve.
PR-FRAG lower 0 The total number of times a trajectory is fragmented (i.e. interrupted during tracking) along the PR curve.
PR-FP lower 0 The total number of false positives along the PR curve.
PR-FN lower 0 The total number of false negatives (missed targets) along the PR curve.

Tracking Challenge Results

Tracking challenge results are presented blow.

MOT system PR-MOTA PR-MOTP PR-MT PR-ML PR-IDS PR-FRAG PR-FP PR-FN Speed(fps) Environment
1 CompACT+GOG 14.2% 37.0% 13.9% 19.9% 3334.6 3172.4 32092.9 180183.8 389.51 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
2 CompACT+CMOT 12.6% 36.1% 16.1% 18.6% 285.3 1516.8 57885.9 167110.8 3.79 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
3 CompACT+H2T 12.4% 35.7% 14.8% 19.4% 852.2 1117.2 51765.7 173899.8 3.02 4x Inter Core i7-3520M @2.90GHz 16GB (C++)
4 R-CNN+DCT 11.7% 38.0% 10.1% 22.8% 758.7 742.9 336561.2 210855.6 0.71 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++)
5 R-CNN+H2T 11.1% 37.3% 14.6% 19.8% 1481.9 1820.8 66137.2 184358.2 2.78 4x Inter Core i7-3520M @2.90GHz 16GB (C++)
5 CompACT+IHTLS 11.1% 36.8% 13.8% 19.9% 953.6 3556.9 53922.3 180422.3 19.79 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
7 R-CNN+CMOT 11.0% 37.0% 15.7% 19.0% 506.2 22551.1 74253.6 177532.6 3.59 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
8 CompACT+DCT 10.8% 37.1% 6.7% 29.3% 141.4 132.4 13226.1 223578.8 2.19 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++)
8 ACF+GOG 10.8% 37.6% 12.2% 22.3% 3950.8 3987.3 45201.5 197094.2 319.29 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
10 R-CNN+GOG 10.0% 38.3% 13.5% 20.1% 7834.5 7401.0 58378.5 192302.7 352.80 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
11 R-CNN+IHTLS 8.3% 38.3% 12.0% 21.4% 1536.4 5954.9 68662.6 199268.8 11.96 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
12 ACF+H2T 8.2% 36.5% 13.1% 21.3% 1122.8 1445.8 71567.4 189649.1 1.08 4x Inter Core i7-3520M @2.90GHz 16GB (C++)
13 ACF+DCT 7.9% 37.9% 4.8% 34.4% 108.1 101.4 13059.7 251166.4 1.29 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++)
14 ACF+CMOT 7.8% 36.8% 14.3% 20.7% 418.3 2161.7 81401.4 183400.2 3.12 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
15 ACF+IHTLS 6.6% 37.4% 11.5% 22.4% 1243.1 4723.0 72757.5 198673.5 5.09 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
16 DPM+GOG 5.5% 28.2% 4.1% 27.7% 1873.9 1988.5 38957.6 230126.6 476.52 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
17 CompACT+CEM 5.1% 35.2% 3.0% 35.3% 267.9 352.3 12341.2 260390.4 4.62 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
18 ACF+CEM 4.5% 35.9% 2.9% 37.1% 265.4 366.0 15180.3 270643.2 3.74 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
19 DPM+CEM 3.3% 27.9% 1.3% 37.8% 265.0 317.1 13888.7 270718.5 4.49 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
20 DPM+DCT 2.7% 29.3% 0.5% 42.7% 72.2 68.8 7785.8 280762.2 2.85 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++)
20 R-CNN+CEM 2.7% 35.5% 2.3% 34.1% 778.9 1080.4 34768.9 269043.8 5.40 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
22 DPM+H2T -0.7% 28.8% 2.1% 28.4% 1738.8 1525.6 71631.0 236520.9 1.77 4x Inter Core i7-3520M @2.90GHz 16GB (C++)
23 DPM+IHTLS -3.0% 27.9% 1.1% 29.8% 1583.6 4153.5 79197.5 244232.8 7.94 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)
24 DPM+CMOT -3.4% 28.4% 5.1% 26.6% 447.5 1040.5 104768.3 221991.7 4.48 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab)