This table describes the evaluation metrics used in the DETRAC-tracking benchmark.
| Measurement | Better | Perfect | Description |
|---|---|---|---|
| PR-MOTA | higher | 100% | Multiple Object Tracking Accuracy along the PR curve. This measure combines three error sources: false positives, missed targets and identity switches. |
| PR-MOTP | higher | 100% | Multiple Object Tracking Precision along the PR curve. The misalignment between the annotated and the predicted bounding boxes. |
| PR-MT | higher | 100% | Mostly tracked targets along the PR curve. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span. |
| PR-ML | lower | 0% | Mostly lost targets along the PR curve. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span. |
| PR-IDS | lower | 0 | The total number of identity switches along the PR curve. |
| PR-FRAG | lower | 0 | The total number of times a trajectory is fragmented (i.e. interrupted during tracking) along the PR curve. |
| PR-FP | lower | 0 | The total number of false positives along the PR curve. |
| PR-FN | lower | 0 | The total number of false negatives (missed targets) along the PR curve. |
Tracking challenge results are presented blow.
| MOT system | PR-MOTA | PR-MOTP | PR-MT | PR-ML | PR-IDS | PR-FRAG | PR-FP | PR-FN | Speed(fps) | Environment | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | frcnn+6thAI | 30.7% | 37.4% | 28.7% | 23.2% | 143.3 | 1183.1 | 13387.9 | 195193.9 | - | Intel i5 @2.30GHz 8GB |
| Sumit Yadav, Rajesh Venkatachalam | |||||||||||
| 2 | Mask R-CNN+V-IOU | 30.7% | 37.0% | 32.0% | 22.6% | 162.6 | 286.2 | 18046.2 | 179191.2 | 359.18 | (Python) |
| Erik Bochinski, Tobias Senst and Thomas Sikora, Extending IOU Based Multi-Object Tracking by Visual Information, In IEEE AVSS 2018. | |||||||||||
| 3 | EB+Kalman-IOUT | 21.1% | 28.6% | 21.9% | 17.6% | 462.2 | 721.1 | 19046.8 | 159178.3 | - | AMD 1950X @3.4GHz 32GB (Python) |
| Siyuan Chen, Chenhui Shao. Dept of Mechanical Science and Engineering, University of Illinois Urbana-Champaign | |||||||||||
| 4 | EB+DAN | 20.2% | 26.3% | 14.5% | 18.1% | 518.2 | - | 9747.8 | 135978.1 | 6.3 | Nvidia GTX Titan (Python) |
| Shijie Sun, Naveed Akhtar, Huansheng Song, Ajmal Mian, Mubarak Shah. Deep Affinity Network for Multiple Object Tracking, T-PAMI 2019. | |||||||||||
| 5 | CompACT+FAMNet | 19.8% | 36.7% | 17.1% | 18.2% | 617.4 | 970.2 | 14988.6 | 164432.6 | - | Nvidia GTX Titan (Python) |
| Peng Chu and Haibin Ling. FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking, in IEEE ICCV 2019. | |||||||||||
| 6 | EB+IOUT | 19.4% | 28.9% | 17.7% | 18.4% | 2311.3 | 2445.9 | 14796.5 | 171806.8 | 6902.07 | Intel i7-6700 @3.40GHz 32GB (Python) |
| E. Bochinski, V. Eiselein, T. Sikora. High-Speed Tracking-by-Detection Without Using Image Information. In International Workshop on Traffic and Street Surveillance for Safety and Security at IEEE AVSS 2017. | |||||||||||
| 7 | R-CNN+IOUT | 16.0% | 38.3% | 13.8% | 20.7% | 5029.4 | 5795.7 | 22535.1 | 193041.9 | 100842.32 | Intel i7-6700 @3.40GHz 32GB (Python) |
| E. Bochinski, V. Eiselein, T. Sikora. High-Speed Tracking-by-Detection Without Using Image Information. In International Workshop on Traffic and Street Surveillance for Safety and Security at IEEE AVSS 2017, 2017. | |||||||||||
| 8 | CompACT+GOG | 14.2% | 37.0% | 13.9% | 19.9% | 3334.6 | 3172.4 | 32092.9 | 180183.8 | 389.51 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 9 | CompACT+CMOT | 12.6% | 36.1% | 16.1% | 18.6% | 285.3 | 1516.8 | 57885.9 | 167110.8 | 3.79 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 10 | CompACT+H2T | 12.4% | 35.7% | 14.8% | 19.4% | 852.2 | 1117.2 | 51765.7 | 173899.8 | 3.02 | 4x Inter Core i7-3520M @2.90GHz 16GB (C++) |
| 11 | R-CNN+DCT | 11.7% | 38.0% | 10.1% | 22.8% | 758.7 | 742.9 | 336561.2 | 210855.6 | 0.71 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++) |
| 12 | R-CNN+H2T | 11.1% | 37.3% | 14.6% | 19.8% | 1481.9 | 1820.8 | 66137.2 | 184358.2 | 2.78 | 4x Inter Core i7-3520M @2.90GHz 16GB (C++) |
| 13 | CompACT+IHTLS | 11.1% | 36.8% | 13.8% | 19.9% | 953.6 | 3556.9 | 53922.3 | 180422.3 | 19.79 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 14 | R-CNN+CMOT | 11.0% | 37.0% | 15.7% | 19.0% | 506.2 | 22551.1 | 74253.6 | 177532.6 | 3.59 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 15 | CompACT+DCT | 10.8% | 37.1% | 6.7% | 29.3% | 141.4 | 132.4 | 13226.1 | 223578.8 | 2.19 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++) |
| 16 | ACF+GOG | 10.8% | 37.6% | 12.2% | 22.3% | 3950.8 | 3987.3 | 45201.5 | 197094.2 | 319.29 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 17 | R-CNN+GOG | 10.0% | 38.3% | 13.5% | 20.1% | 7834.5 | 7401.0 | 58378.5 | 192302.7 | 352.80 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 18 | R-CNN+IHTLS | 8.3% | 38.3% | 12.0% | 21.4% | 1536.4 | 5954.9 | 68662.6 | 199268.8 | 11.96 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 19 | ACF+H2T | 8.2% | 36.5% | 13.1% | 21.3% | 1122.8 | 1445.8 | 71567.4 | 189649.1 | 1.08 | 4x Inter Core i7-3520M @2.90GHz 16GB (C++) |
| 20 | ACF+DCT | 7.9% | 37.9% | 4.8% | 34.4% | 108.1 | 101.4 | 13059.7 | 251166.4 | 1.29 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++) |
| 21 | ACF+CMOT | 7.8% | 36.8% | 14.3% | 20.7% | 418.3 | 2161.7 | 81401.4 | 183400.2 | 3.12 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 22 | ACF+IHTLS | 6.6% | 37.4% | 11.5% | 22.4% | 1243.1 | 4723.0 | 72757.5 | 198673.5 | 5.09 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 23 | DPM+GOG | 5.5% | 28.2% | 4.1% | 27.7% | 1873.9 | 1988.5 | 38957.6 | 230126.6 | 476.52 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 24 | CompACT+CEM | 5.1% | 35.2% | 3.0% | 35.3% | 267.9 | 352.3 | 12341.2 | 260390.4 | 4.62 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 25 | ACF+CEM | 4.5% | 35.9% | 2.9% | 37.1% | 265.4 | 366.0 | 15180.3 | 270643.2 | 3.74 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 26 | DPM+CEM | 3.3% | 27.9% | 1.3% | 37.8% | 265.0 | 317.1 | 13888.7 | 270718.5 | 4.49 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 27 | DPM+DCT | 2.7% | 29.3% | 0.5% | 42.7% | 72.2 | 68.8 | 7785.8 | 280762.2 | 2.85 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab, C++) |
| 28 | R-CNN+CEM | 2.7% | 35.5% | 2.3% | 34.1% | 778.9 | 1080.4 | 34768.9 | 269043.8 | 5.40 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 29 | DPM+H2T | -0.7% | 28.8% | 2.1% | 28.4% | 1738.8 | 1525.6 | 71631.0 | 236520.9 | 1.77 | 4x Inter Core i7-3520M @2.90GHz 16GB (C++) |
| 30 | DPM+IHTLS | -3.0% | 27.9% | 1.1% | 29.8% | 1583.6 | 4153.5 | 79197.5 | 244232.8 | 7.94 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |
| 31 | DPM+CMOT | -3.4% | 28.4% | 5.1% | 26.6% | 447.5 | 1040.5 | 104768.3 | 221991.7 | 4.48 | 4x Inter Core i7-3520M @2.90GHz 16GB (Matlab) |