Multi-Object Tracking with Grayscale Spatial-Temporal Features
Source abstract
Abstract
In recent multiple object tracking (MOT) research, there have not been many traditional methods and optimizations for matching. Most of today’s popular tracking methods are implemented using deep learning. But many monitoring devices do not have high computing power, so real-time tracking via neural networks is difficult. Furthermore, matching takes less time than detection and embedding, but it still takes some time, especially for many targets in a scene. Therefore, in order to solve these problems, we propose a new method by using grayscale maps to obtain spatial-temporal features based on traditional methods. Using this method allows us to directly find the position and region in previous frames of the target and significantly reduce the number of IDs that the target needs to match. At the same time, compared to some end-to-end paradigms, our method can quickly obtain spatial-temporal features using traditional methods, which reduces some calculations. Further, we joined embedding and matching to further reduce the time spent on tracking. Our method reduces the calculations in feature extraction and reduces unnecessary matching in the matching stage. Our method was evaluated on benchmark dataset MOT16, and it achieved great performance; the tracking accuracy metric MOTA reached 46.7%. The tracking FPS reached 17.6, and it ran only on a CPU without GPU acceleration.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Image and Video Retrieval Techniques · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic Information, Qingdao University, Qingdao 266071, China
References (32)
Show source-provided references
1. Redmon, J., Divvala, S., Divvala, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
2. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster r-cnn: Towards real-time object detection with region proposal networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, USA.
3. Liu, W., Anguelov, D., and Erhan, D. (2016, January 11–14). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherland.10.1007/978-3-319-46448-0_2 ↗
4. Bewley, A., Ge, Z., and Ott, L. (2016, January 25–28). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533003 ↗
5. Wang, Z., Zheng, L., and Liu, Y. (2020, January 23–28). Towards real-time multi-object tracking. Proceedings of the European Conference on Computer Vision, Virtual.10.1007/978-3-030-58621-8_7 ↗
6. Bergmann, P., Meinhardt, T., and Leal-Taixe, L. (2019, January 15–20). Tracking without bells and whistles. Proceedings of the IEEE/CVF International Conference on Computer Vision, Long Beach, CA, USA.10.1109/ICCV.2019.00103 ↗
7. Milan, A., Leal-taixé, L., Reid, I., Roth, S., and Schindler, K. (2016). MOT16: A benchmark for multi-object tracking. arXiv.
8. Cao, J., Pang, J., and Weng, X. (2023, January 18–22). Observation-centric sort: Rethinking sort for robust multi-object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00934 ↗
9. Qin, Z., Zhou, S., and Wang, L. (2023, January 18–22). Motiontrack: Learning robust short-term and long-term motions for multi-object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01720 ↗
10. Yu, F., Li, W., and Li, Q. (2016, January 11–14). POI: Multiple object tracking with high performance detection and appearance feature. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-48881-3_3 ↗
11. Szegedy, C., Liu, W., and Jia, Y. (2015, January 7–12). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
12. Dense Neighborhoods on Affinity Graph10.1007/s11263-011-0496-1 ↗
13. LI, Z., Cai, S., and Cai, X. (2021, January 18–22). Multiple Object Tracking with GRU Association and Kalman Prediction. Proceedings of the International Joint Conference on Neural Networks, Virtual.
14. Yang, M., Han, G., and Yan, B. (2024, January 20–27). Hybrid-sort: Weak cues matter for online multi-object tracking. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i7.28471 ↗
15. Lu, Z., Rathod, V., and Votel, R. (2020, January 14–19). Retinatrack: Online single stage joint detection and tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01468 ↗
16. Lin, T., Goyal, P., and Girshick, R. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
17. Multi-Object Tracking: Decoupling Features to Solve the Contradictory Dilemma of Feature Requirements10.1109/TCSVT.2023.3249162 ↗
18. Vaswani, A., Shazeer, N., and Parmar, N. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
19. Meinhardt, T., Kirillov, A., and Leal-taixe, L. (2022, January 18–24). Trackformer: Multi-object tracking with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00864 ↗
20. Carion, N., Massa, F., and Synnaeve, G. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Virtual.10.1007/978-3-030-58452-8_13 ↗
21. Zhang, Y., Wang, T., and Zhang, X. (2023, January 18–22). MOTRv2: Bootstrapping end-to-end multi-object tracking by pretrained object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02112 ↗
22. Zeng, F., Dong, B., and Zhang, Y. (2022, January 23–27). MOTR: End-to-end multiple-object tracking with transformer. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19812-0_38 ↗
23. Ballester, I., Fontán, A., and Civera, J. (June, January 30). DOT: Dynamic Object Tracking for Visual SLAM. Proceedings of the 2021 IEEE International Conference on Robotics and Automation, Xi’an, China.10.1109/ICRA48506.2021.9561452 ↗
24. Online multi-object tracking method based on trajectory mask
25. Saleh, F., Aliakbarian, S., and Gould, S. (2021, January 20–25). Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01410 ↗
26. Fairmot: On the fairness of detection and re-identification in multiple object tracking10.1007/s11263-021-01513-4 ↗
27. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., and Yu, N. (2017, January 22–29). Online Multi-object Tracking Using CNN-Based Single Object Tracker with Spatial-Temporal Attention Mechanism. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.518 ↗
28. Zhu, J., Yang, H., Liu, N., and Kim, M. (2018, January 8–14). Online Multi-Object Tracking with Dual Matching Attention Networks. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01228-1_23 ↗
29. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., and Yu, N. (2020, January 7–12). DASOT: A Unified Framework Integrating Data Association and Single Object Tracking for Online Multi-Object Tracking. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6694 ↗
30. Online multiple pedestrians tracking using deep temporal appearance matching association10.1016/j.ins.2020.10.002 ↗
31. Online multiple object tracking based on fusing global and partial features
32. Sadeghian, A., Alahi, A., and Savarese, S. (2017, January 22–29). Tracking the untrackable: Learning to track multiple cues with long-term dependencies. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.41 ↗