Multi-Scale Ensemble Learning for Thermal Image Enhancement
Source abstract
Abstract
In this study, we propose a multi-scale ensemble learning method for thermal image enhancement in different image scale conditions based on convolutional neural networks. Incorporating the multiple scales of thermal images has been a tricky task so that methods have been individually trained and evaluated for each scale. However, this leads to the limitation that a network properly operates on a specific scale. To address this issue, a novel parallel architecture leveraging the confidence maps of multiple scales have been introduced to train a network that operates well in varying scale conditions. The experimental results show that our proposed method outperforms the conventional thermal image enhancement methods. The evaluation is presented both quantitatively and qualitatively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Image and Signal Denoising Methods · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Agency for Defense Development, Daejeon 34068, Korea
School of Artificial Intelligence, Yong In University, Yongin 17092, Korea
Funding
National Research Foundation of Korea
References (35)
Show source-provided references
1. Kwaśniewska, A., Rumiński, J., and Rad, P. (2017, January 17–19). Deep features class activation map for thermal face detection and tracking. Proceedings of the 2017 10th international conference on human system interactions (HSI), Ulsan, Korea.10.1109/HSI.2017.8004993 ↗
2. A hybrid framework to evaluate breast abnormality using infrared thermal images10.1109/MCE.2019.2923926 ↗
3. Tong, K., Wang, Z., Si, L., Tan, C., and Li, P. (2020). A Novel Pipeline Leak Recognition Method of Mine Air Compressor Based on Infrared Thermal Image Using IFA and SVM. Appl. Sci., 10.10.3390/app10175991 ↗
4. Baek, J., Hong, S., Kim, J., and Kim, E. (2017). Efficient Pedestrian Detection at Nighttime Using a Thermal Camera. Sensors, 17.10.3390/s17081850 ↗
5. Filippini, C., Perpetuini, D., Cardone, D., Chiarelli, A.M., and Merla, A. (2020). Thermal infrared imaging-based affective computing and its application to facilitate human robot interaction: A review. Appl. Sci., 10.10.3390/app10082924 ↗
6. Andoga, R., Fozo, L., Schrötter, M., Češkovič, M., Szabo, S., Breda, R., and Schreiner, M. (2019). Intelligent thermal imaging-based diagnostics of turbojet engines. Appl. Sci., 9.10.3390/app9112253 ↗
7. B5G and explainable deep learning assisted healthcare vertical at the edge: COVID-I9 perspective10.1109/MNET.011.2000353 ↗
8. Opportunities and challenges for the building monitoring systems in the age-pandemic of COVID-19: Review and prospects
9. Taylor, W., Abbasi, Q.H., Dashtipour, K., Ansari, S., Shah, S.A., Khalid, A., and Imran, M.A. (2020). A Review of the State of the Art in Non-Contact Sensing for COVID-19. Sensors, 20.10.3390/s20195665 ↗
10. Choi, Y., Kim, N., Hwang, S., and Kweon, I.S. (2016, January 9–14). Thermal Image Enhancement using Convolutional Neural Network. Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.10.1109/IROS.2016.7759059 ↗
11. Brightness-based convolutional neural network for thermal image enhancement10.1109/ACCESS.2017.2769687 ↗
12. Rivadeneira, R.E., Suárez, P.L., Sappa, A.D., and Vintimilla, B.X. (2019, January 27–29). Thermal Image Superresolution Through Deep Convolutional Neural Network. Proceedings of the International Conference on Image Analysis and Recognition, Waterloo, ON, Canada.10.1007/978-3-030-27272-2_37 ↗
13. Gupta, H., and Mitra, K. (2020). Pyramidal Edge-maps based Guided Thermal Super-resolution. arXiv.10.1007/978-3-030-67070-2_42 ↗
14. Rivadeneira, R.E., Sappa, A.D., and Vintimilla, B.X. (2020, January 27–29). Thermal Image SUPER-Resolution: A Novel Architecture and Dataset. Proceedings of the VISIGRAPP (4: VISAPP), Valleta, Malta.10.5220/0009173601110119 ↗
15. Cascarano, P., Corsini, F., Gandolfi, S., Piccolomini, E.L., Mandanici, E., Tavasci, L., and Zama, F. (2020). Super-resolution of thermal images using an automatic total variation based method. Remote Sens., 12.10.3390/rs12101642 ↗
16. Infrared image enhancement using adaptive trilateral contrast enhancement10.1016/j.patrec.2014.09.011 ↗
17. Zeng, Q., Qin, H., Yan, X., Yang, S., and Yang, T. (2018). Single infrared image-based stripe nonuniformity correction via a two-stage filtering method. Sensors, 18.10.3390/s18124299 ↗
18. Dual-Branch Structured De-Striping Convolution Network Using Parametric Noise Model10.1109/ACCESS.2020.3019057 ↗
19. Liu, Y., Wang, Z., Si, L., Zhang, L., Tan, C., and Xu, J. (2017). A non-reference image denoising method for infrared thermal image based on enhanced dual-tree complex wavelet optimized by fruit fly algorithm and bilateral filter. Appl. Sci., 7.10.3390/app7111190 ↗
20. Brightness preserving dynamic histogram equalization for image contrast enhancement10.1109/TCE.2007.4429280 ↗
21. Infrared image enhancement through contrast enhancement by using multiscale new top-hat transform10.1016/j.infrared.2010.12.001 ↗
22. Thermal image enhancement using bi-dimensional empirical mode decomposition in combination with relevance vector machine for rotating machinery fault diagnosis10.1016/j.ymssp.2013.02.001 ↗
23. Single infrared image enhancement using a deep convolutional neural network10.1016/j.neucom.2018.11.081 ↗
24. Berg, A., Ahlberg, J., and Felsberg, M. (2015, January 25–28). A thermal object tracking benchmark. Proceedings of the 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Karlsruhe, Germany.10.1109/AVSS.2015.7301772 ↗
25. Multi-modal RGB–Depth–Thermal Human Body Segmentation10.1007/s11263-016-0901-x ↗
26. Portmann, J., Lynen, S., Chli, M., and Siegwart, R. (June, January 31). People detection and tracking from aerial thermal views. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907094 ↗
27. Morris, N.J., Avidan, S., Matusik, W., and Pfister, H. (2007, January 17–22). Statistics of infrared images. Proceedings of the Computer Vision and Pattern Recognition, CVPR’07, Minneapolis, MN, USA.10.1109/CVPR.2007.383003 ↗
28. Hwang, S., Park, J., Kim, N., Choi, Y., and So Kweon, I. (2015, January 7–12). Multispectral pedestrian detection: Benchmark dataset and baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298706 ↗
29. KAIST multi-spectral day/night data set for autonomous and assisted driving10.1109/TITS.2018.2791533 ↗
30. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
31. Yamanaka, J., Kuwashima, S., and Kurita, T. (2017, January 14–18). Fast and accurate image super resolution by deep CNN with skip connection and network in network. Proceedings of the International Conference on Neural Information Processing, Guangzhou, China.10.1007/978-3-319-70096-0_23 ↗
32. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
33. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 13–16). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.123 ↗
34. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
35. Image super-resolution via sparse representation10.1109/TIP.2010.2050625 ↗