journal article·2025-03

Reconstructing hyperspectral images of textiles from a single RGB image utilizing the multihead self-attention mechanism

Jianxin Zhang, Jin Ma, Miao Qian, Ming Wang

Textile Research Journal· Volume 95· Issue 5-6
Publisher page

Abstract

Hyperspectral images possess abundant information and play a pivotal role in enhancing the accuracy of color difference detection in textiles. However, traditional hyperspectral imaging methods necessitate costly equipment and intricate operational procedures. A novel deep learning model based on a multihead attention mechanism was proposed in this article to facilitate the extensive application of hyperspectral imaging technology in textile quality inspection. This model enabled the reconstruction of the hyperspectral information of plain weave textiles from a single RGB image. In this model, encoder-decoder architecture and pyramid pooling convolutional operations were employed to integrate multiscale features of plain weave cotton-linen textiles. This could capture details and contextual information in textile images more precisely, enhancing the accuracy of hyperspectral image reconstruction. Simultaneously, an attention mechanism was introduced to increase the model’s receptive field and improve its focus on key regions in the input image and feature maps. This resulted in a reduced weighting of redundant information during network learning, leading to an improved feature extraction capability of the network. Through these methods, successful reconstructions of plain weave textiles hyperspectral information from a single RGB image was achieved. Quantitative and qualitative tests were conducted on two datasets, namely, the NTIRE 2020 dataset and a self-made textile dataset, to evaluate the performance of the proposed method. The approach proposed in this article exhibited promising results on both datasets. Specifically, the reconstructed textile hyperspectral images achieved a root mean square error of 0.0344, a peak signal-to-noise ratio of 29.945, a spectral angle mapper of 3.753, and a structural similarity index measure of 0.955 on the textile dataset. In the reconstructed hyperspectral colorimetric experiment, the maximum value of average color difference was 2.641. These results demonstrate that the method can meet the requirements for textile color measurement applications.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Color Science and Applications · Remote-Sensing Image Classification · Industrial Vision Systems and Defect Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Mechanical Engineering, Zhejiang Sci-Tech University, China

School of Information Science and Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China

Funding

the Key R&D projects of Science and Technology Department of Zhejiang Province · National Natural Science Foundation of China

References (24)

Show source-provided references
1. 10.1177/004051751988395310.1177/0040517519883953 ↗
2. 10.1177/096033601668794910.1177/0960336016687949 ↗
3. Parmar M Lansel S Wandell BA. Spatio-spectral reconstruction of the multispectral datacube using sparse recovery. In: 21st ACM Conference on Computer and Communications Security Scottsdale AZ United States 3–7 November 2014 pp. 19–34.
4. Arad B Ben-Shahar O. Sparse recovery of hyperspectral signal from natural RGB images. In: Computer vision-ECCV 2016: 14th European conference Amsterdam The Netherlands 11–14 October 2016 Proceedings Part VII 14. Springer International Publishing pp. 19–34.10.1007/978-3-319-46478-7_2 ↗
5. Aeschbacher J Wu J Timofte R. In defense of shallow learned spectral reconstruction from RGB images. In: Proceedings of the IEEE international conference on computer vision workshops (ICCVW) Venice Italy 22–29 October 2017 pp. 471–479.
6. Li F, Li C. Spectral reflectance reconstruction based on camera raw RGB using weighted third-order polynomial and Wiener estimation. Spectrosc Spect Anal 2021; 41: 3281–3285.
7. 10.1364/OE.27.03492110.1364/OE.27.034921 ↗
8. Han XH Shi B Zheng Y. Residual HSRCNN: Residual hyper-spectral reconstruction CNN from an RGB Image. In 24th International Conference on Pattern Recognition (ICPR) Beijing China 20–24 August 2018 pp. 2664–2669.10.1109/ICPR.2018.8545634 ↗
9. Stiebei T Koppers S Seltsam P et al. Reconstructing spectral images from RGB-images using a convolutional neural network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Salt Lake City UT USA 18–22 June 2018 pp. 948–953.10.1109/CVPRW.2018.00140 ↗
10. Yan Y Zhang L Li J et al. Accurate spectral super-resolution from single RGB image using multi-scale CNN. In: Pattern Recognition and Computer Vision Guangzhou China 23–26 November 2018 pp. 206–217.10.1007/978-3-030-03335-4_18 ↗
11. Shi Z Chen C Xiong Z et al. HSCNN+: Advanced CNN-based hyperspectral recovery from RGB images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Salt Lake City UT USA 18–22 June 2018; pp. 939–947.
12. Zhang L Lang Z Wang P et al. Pixel-aware deep function-mixture network for spectral super-resolution. In: Proceedings of the AAAI Conference on Artificial Intelligence New York NY United states 7–12 February 2020; 34: 12821–12828.10.1609/aaai.v34i07.6978 ↗
13. Lee SH Park MJ Kim JO. Progressive deep network with channel back-projection for hyperspectral recovery from RGB. In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) Auckland New Zealand 07–10 December 2020 pp. 1257–1261.
14. Alvarez-Gila A Van De Weijer J Garrote E. Adversarial networks for spatial context-aware spectral image reconstruction from RGB. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW) Venice Italy 22–29 October 2017 pp. 480–490.10.1109/ICCVW.2017.64 ↗
15. Peng H Chen X Zhao J. Residual pixel attention network for spectral reconstruction from RGB images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Seattle WA USA 14–19 June 2020 pp. 486–487.10.1109/CVPRW50498.2020.00251 ↗
16. Nathan DS Uma K Vinothini DS. Light weight residual dense attention net for spectral reconstruction from RGB images. arXiv. 2020. https://arxiv.org/abs/2004.06930
17. Li J Wu C Song R et al. Adaptive weighted attention network with camera spectral sensitivity prior for spectral reconstruction from RGB images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Seattle WA USA 14–19 June 2020 pp. 462–463.10.1109/CVPRW50498.2020.00239 ↗
18. Li J Du S Wu C et al. DRCR net: Dense residual channel re-calibration network with non-local purification for spectral super resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPRW) New Orleans LA USA 19–20 June 2022 pp. 1259–1268.10.1109/CVPRW56347.2022.00132 ↗
19. 10.1109/TCI.2018.285544510.1109/TCI.2018.2855445 ↗
20. Shaw P Uszkoreit J Vaswani A. Self-attention with relative position representations. arXiv 12 Apr 2018; URL: https://arxiv.org/abs/1803.0215510.18653/v1/N18-2074 ↗
21. Liu Z Lin Y Cao Y et al. Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Montreal QC Canada 10–17 October 2021 pp. 10012–10022.
22. 10.1177/0040517522111810510.1177/00405175221118105 ↗
23. 10.3390/s2008242610.3390/s20082426 ↗
24. 10.1002/col.2273810.1002/col.22738 ↗