journal article·2025-06-05Open access

Dual Branch Encoding Feature Aggregation for Cloud and Cloud Shadow Detection of Remote Sensing Image

Naikang Shi, Haifeng Lin, Huiwen Ji, Min Xia

Applied Sciences· Volume 15· Issue 11

Abstract

Cloud detection is a critical preprocessing step for optical remote sensing imagery. However, traditional CNN-based methods have limitations in global feature modeling, while Transformer models, despite their strong global modeling capability, struggle to capture fine-grained local details effectively. To tackle these challenges, this study introduces a dual-path neural network framework that synergistically combines convolutional neural networks (CNNs) and architectures. By capitalizing on their complementary strengths, this work proposed a dual-branch feature extraction architecture that utilizes two different feature aggregation modes to effectively aggregate features of CNN and Transformer at different levels. Specifically, two novel modules are introduced: the Dual-branch Lightweight Aggregation Module (DLAM), which fuses CNN and Transformer features in the early encoding stage and emphasizes key information through a feature weight allocation mechanism; and the Dual-branch Attention Aggregation Module (DAAM), which further integrates local and global features in the late encoding stage, improving the model’s differentiation performance between cloud and cloud shadow areas. The collaboration between DLAM and DAAM enables the model to efficiently learn multi-scale and spatially hierarchical information, thereby improving detection performance. The experimental findings validate the superior performance of our model over state-of-the-art methods on diverse remote sensing datasets, attaining enhanced accuracy in cloud detection.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Image Fusion Techniques · Image Enhancement Techniques · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Science, China University of Mining & Technology, Beijing 100083, China

College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China

Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, B-DAT, Nanjing University of Information Science and Technology, Nanjing 210044, China

Funding

undergraduate Innovative program · National Natural Science Foundation of PR China

References (52)

Show source-provided references
1. Automatic cloud-shadow removal from flood/standing water maps using MSG/SEVIRI imagery10.1080/01431161.2013.792969 ↗
2. Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: An algorithm designed specifically for monitoring land cover change10.1016/j.rse.2014.06.012 ↗
3. Automated masking of cloud and cloud shadow for forest change analysis using Landsat images10.1080/01431160903369642 ↗
4. Retrieval of cloud cover from satellite imagery data: A statistical approach10.1029/JD090iD07p12960 ↗
5. Object-based cloud and cloud shadow detection in Landsat imagery10.1016/j.rse.2011.10.028 ↗
6. Improvement and expansion of the Fmask algorithm: Cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 images10.1016/j.rse.2014.12.014 ↗
7. Improving Fmask cloud and cloud shadow detection in mountainous area for Landsats 4–8 images10.1016/j.rse.2017.07.002 ↗
8. Improvement of the Fmask algorithm for Sentinel-2 images: Separating clouds from bright surfaces based on parallax effects10.1016/j.rse.2018.04.046 ↗
9. Fmask 4.0: Improved cloud and cloud shadow detection in Landsats 4–8 and Sentinel-2 imagery10.1016/j.rse.2019.05.024 ↗
10. Cheng, P., Xia, M., Wang, D., Lin, H., and Zhao, Z. (2025). Transformer Self-Attention Change Detection Network with Frozen Parameters. Appl. Sci., 15.10.3390/app15063349 ↗
11. Cloud Detection of MODIS Satellite Images Based on Dynamical Cluster
12. A new Landsat 8 cloud discrimination algorithm using thresholding tests10.1080/01431161.2018.1506183 ↗
13. Characterization of the Landsat-7 ETM+ automated cloud-cover assessment (ACCA) algorithm10.14358/PERS.72.10.1179 ↗
14. An automatic method for screening clouds and cloud shadows in optical satellite image time series in cloudy regions10.1016/j.rse.2018.05.024 ↗
15. Candra, D.S., Phinn, S., and Scarth, P. (2017, January 7–10). Cloud and cloud shadow removal of Landsat 8 images using Multitemporal Cloud Removal method. Proceedings of the 2017 6th International Conference on Agro-Geoinformatics, Fairfax, VA, USA.10.1109/Agro-Geoinformatics.2017.8047007 ↗
16. Lin, J., Huang, T.Z., Zhao, X.L., Ding, M., Chen, Y., and Jiang, T.X. (2021, January 11–16). A Blind Cloud/Shadow Removal Strategy for Multi-Temporal Remote Sensing Images. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Brussels, Belgium.10.1109/IGARSS47720.2021.9554515 ↗
17. Method for detecting cloud in satellite remote sensing image based on texture
18. Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery10.1016/j.rse.2017.01.026 ↗
19. Hollstein, A., Segl, K., Guanter, L., Brell, M., and Enesco, M. (2016). Ready-to-use methods for the detection of clouds, cirrus, snow, shadow, water and clear sky pixels in Sentinel-2 MSI images. Remote Sens., 8.10.3390/rs8080666 ↗
20. Construction of cloud-shadow-water mask based on Random Forests algorithm
21. Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing10.3390/rs6064907 ↗
22. Jiang, S., Lin, H., Ren, H., Hu, Z., Weng, L., and Xia, M. (2024). MDANet: A High-Resolution City Change Detection Network Based on Difference and Attention Mechanisms under Multi-Scale Feature Fusion. Remote Sens., 16.10.3390/rs16081387 ↗
23. HyCloudX: A Multibranch Hybrid Segmentation Network With Band Fusion for Cloud/Shadow10.1109/JSTARS.2024.3374233 ↗
24. Zhan, Z., Ren, H., Xia, M., Lin, H., Wang, X., and Li, X. (2024). AMFNet: Attention-Guided Multi-Scale Fusion Network for Bi-Temporal Change Detection in Remote Sensing Images. Remote Sens., 16.10.3390/rs16101765 ↗
25. TSI-Siamnet: A Siamese network for cloud and shadow detection based on time-series cloudy images10.1016/j.isprsjprs.2024.05.022 ↗
26. Permutohedral Refined UNet: Bilateral Feature-Scalable Segmentation Network for Edge-Precise Cloud and Shadow Detection10.1109/JSTARS.2024.3383446 ↗
27. CSDUNet: Automatic Cloud and Shadow Detection from Satellite Images Based on Modified U-Net10.1007/s12524-024-01903-4 ↗
28. Wang, J., Li, Y., Fan, X., Zhou, X., and Wu, M. (2024). MRFA-Net: Multi-Scale Receptive Feature Aggregation Network for Cloud and Shadow Detection. Remote Sens., 16.10.3390/rs16081456 ↗
29. CloudFU-Net: A Fine-Grained Segmentation Method for Ground-Based Cloud Images Based on an Improved Encoder–Decoder Structure
30. SSM-CloudNet: State space model-based cloud and cloud shadow detection network for Sentinel-2 imagery
31. ANNet: Asymmetric Nested Network for Real-Time Cloud Detection in Remote Sensing
32. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, Proceedings of the Annual Conference on Neural Information Processing Systems 2017, Long Beachh, CA, USA, 4–9 December 2017, Curran Associates Inc.
33. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
34. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.
35. SegFormer: Simple and efficient design for semantic segmentation with transformers
36. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
37. A Lightweight Cloud and Cloud Shadow Detection Transformer with Prior-Knowledge Guidance
38. CSDFormer: A cloud and shadow detection method for landsat images based on transformer
39. Gu, H., Gu, G., Liu, Y., Lin, H., and Xu, Y. (2024). Multi-Branch Attention Fusion Network for Cloud and Cloud Shadow Segmentation. Remote Sens., 16.10.3390/rs16132308 ↗
40. Du, W., Fan, Z., Yan, Y., Yu, R., and Liu, J. (2024). AFMUNet: Attention Feature Fusion Network Based on a U-Shaped Structure for Cloud and Cloud Shadow Detection. Remote Sens., 16.10.3390/rs16091574 ↗
41. Liang, K., Yang, G., Zuo, Y., Chen, J., Sun, W., Meng, X., and Chen, B. (2024). A Novel Method for Cloud and Cloud Shadow Detection Based on the Maximum and Minimum Values of Sentinel-2 Time Series Images. Remote Sens., 16.10.3390/rs16081392 ↗
42. Dual-attention-guided multiscale feature aggregation network for remote sensing image change detection10.1109/JSTARS.2024.3362370 ↗
43. FTA-Net: Frequency-Temporal-Aware Network for Remote Sensing Change Detection10.1109/JSTARS.2025.3525595 ↗
44. Bitemporal Attention Sharing Network for Remote Sensing Image Change Detection10.1109/JSTARS.2024.3400925 ↗
45. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
46. Dong, X., Bao, J., Chen, D., Zhang, W., Yu, N., Yuan, L., Chen, D., and Guo, B. (2022, January 18–24). CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01181 ↗
47. Peng, Z., Huang, W., Gu, S., Xie, L., Wang, Y., Jiao, J., and Ye, Q. (2021, January 10–17). Conformer: Local features coupling global representations for visual recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00042 ↗
48. Chen, Y., Dai, X., Chen, D., Liu, M., Dong, X., Yuan, L., and Liu, Z. (2022, January 18–24). Mobile-former: Bridging mobilenet and transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00520 ↗
49. Strudel, R., Garcia, R., Laptev, I., and Schmid, C. (2021, January 10–17). Segmenter: Transformer for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00717 ↗
50. Wang, W., Xie, E., Li, X., Fan, D.P., Song, K., Liang, D., Lu, T., Luo, P., and Shao, L. (2021, January 10–17). Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00061 ↗
51. Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., and Zhang, L. (2021, January 10–17). CVT: Introducing Convolutions to Vision Transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00009 ↗
52. Early convolutions help transformers see better