journal article·2024-07-18Open access

CellGAN: Generative Adversarial Networks for Cellular Microscopy Image Recognition with Integrated Feature Completion Mechanism

Xiangle Liao, Wenlong Yi

Applied Sciences· Volume 14· Issue 14

Abstract

In response to the challenges of high noise, high adhesion, and a low signal-to-noise ratio in microscopic cell images, as well as the difficulty of existing deep learning models such as UNet, ResUNet, and SwinUNet in segmenting images with clear boundaries and high-resolution, this study proposes a CellGAN semantic segmentation method based on a generative adversarial network with a Feature Completion Mechanism. This method incorporates a Transformer to supplement long-range semantic information. In the self-attention module of the Transformer generator, bilinear interpolation for feature completion is introduced, reducing the computational complexity of self-attention to O(n). Additionally, two-dimensional relative positional encoding is employed in the self-attention mechanism to supplement positional information and facilitate position recovery. Experimental results demonstrate that this method outperforms ResUNet and SwinUNet in segmentation performance on rice leaf cell, MuNuSeg, and Nucleus datasets, achieving up to 23.45% and 19.90% improvements in the Intersection over Union and Similarity metrics, respectively. This method provides an automated and efficient analytical tool for cell biology, enabling more accurate segmentation of cell images, and contributing to a deeper understanding of cellular structure and function.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: AI in cancer detection · Cell Image Analysis Techniques · Generative Adversarial Networks and Image Synthesis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Software, Jiangxi Agricultural University, Nanchang 330045, China

Funding

National Natural Science Foundation of China · Natural Science Foundation of Jiangxi Province · National College Student Innovation and Entrepreneurship Training Program

References (46)

Show source-provided references
1. Immune cells within the tumor microenvironment: Biological functions and roles in cancer immunotherapy10.1016/j.canlet.2019.11.009 ↗
2. Poole, J.J., and Mostaço-Guidolin, L.B. (2021). Optical Microscopy and the Extracellular Matrix Structure: A Review. Cells, 10.10.3390/cells10071760 ↗
3. Magazzù, A., and Marcuello, C. (2023). Investigation of Soft Matter Nanomechanics by Atomic Force Microscopy and Optical Tweezers: A Comprehensive Review. Nanomaterials, 13.10.3390/nano13060963 ↗
4. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes10.1038/s41592-021-01155-x ↗
5. Spatial components of molecular tissue biology10.1038/s41587-021-01182-1 ↗
6. Machine learning techniques for biomedical image segmentation: An overview of technical aspects and introduction to state-of-art applications10.1002/mp.13649 ↗
7. An Overview of Segmentation Algorithms for the Analysis of Anomalies on Medical Images
8. DeepCell Kiosk: Scaling deep learning–enabled cellular image analysis with Kubernetes10.1038/s41592-020-01023-0 ↗
9. Cellpose: A generalist algorithm for cellular segmentation10.1038/s41592-020-01018-x ↗
10. Mondal, A.K., Agarwal, A., Dolz, J., and Desrosiers, C. (2019). Revisiting CycleGAN for semi-supervised segmentation. arXiv.
11. Deep learning architectures for semantic segmentation and automatic estimation of severity of foliar symptoms caused by diseases or pests
12. An image-based system for locating pruning points in apple trees using instance segmentation and RGB-D images10.1016/j.biosystemseng.2023.11.006 ↗
13. Qian, L., Zhou, X., Li, Y., and Hu, Z. (2022). Unet#: A Unet-like redesigning skip connections for medical image segmentation. arXiv.
14. DeepLab V3+ Based Semantic Segmentation of COVID-19 Lesions in Computed Tomography Images
15. Evaluation of UNet and UNet++ architectures in high resolution image change detection applications
16. An Res-Unet method for pulmonary artery segmentation of CT images10.1088/1742-6596/1924/1/012018 ↗
17. Luc, P., Couprie, C., Chintala, S., and Verbeek, J. (2016). Semantic Segmentation using Adversarial Networks. arXiv.
18. Ramwala, O.A., Dhakecha, S.A., Ganjoo, A., Visiya, D., and Sarvaiya, J.N. (2021, January 1–3). Leveraging Adversarial Training for Efficient Retinal Vessel Segmentation. Proceedings of the 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Pitesti, Romania.10.1109/ECAI52376.2021.9515093 ↗
19. Tato, A., and Nkambou, R. (2024, July 07). Improving Adam Optimizer. Available online: https://openreview.net/pdf?id=HJfpZq1DM.
20. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired imageto-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
21. Application of generative adversarial networks (GAN) for ophthalmology image domains: A survey10.1186/s40662-022-00277-3 ↗
22. Retinal vessel segmentation based on task-driven generative adversarial network10.1049/iet-ipr.2020.1032 ↗
23. Multi-scale receptive fields: Graph attention neural network for hyperspectral image classification10.1016/j.eswa.2023.119858 ↗
24. Retinal vessel segmentation combined with generative adversarial networks and Dense U-Net10.1109/ACCESS.2020.3033273 ↗
25. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes10.1109/TMI.2018.2845918 ↗
26. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
27. Attention gated networks: Learning to leverage salient regions in medical images10.1016/j.media.2019.01.012 ↗
28. Focusnetv2: Imbalanced large and small organ segmentation with adversarial shape constraint for head and neck Ct images10.1016/j.media.2020.101831 ↗
29. Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv.
30. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2024, July 07). Attention is all you need. Adv. Neural Inf. Process. Syst., Available online: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
32. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020, January 16–20). Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online.10.18653/v1/2020.emnlp-demos.6 ↗
33. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
34. Xiao, X., Lian, S., Luo, Z., and Li, S. (2018, January 19–21). Weighted res-unet for high-quality retina vessel segmentation. Proceedings of the 2018 9th International Conference on Information Technology in Medicine and Education (ITME), Hangzhou, China.10.1109/ITME.2018.00080 ↗
35. Efficient training of visual transformers with small datasets
36. Exploiting multi-scale parallel self-attention and local variation via dual-branch transformer-cnn structure for face super-resolution10.1109/TMM.2023.3301225 ↗
37. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2022, January 23–27). Swin-unet: Unet-like pure transformer for medical image segmentation. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-25066-8_9 ↗
38. Cao, Y.H., Yu, H., and Wu, J. (2022, January 23–27). Training vision transformers with only 2040 images. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19806-9_13 ↗
39. Salehi, A.W., Khan, S., Gupta, G., Alabduallah, B.I., Almjally, A., Alsolai, H., Siddiqui, T., and Mellit, A. (2023). A study of CNN and transfer learning in medical imaging: Advantages, challenges, future scope. Sustainability, 15.10.3390/su15075930 ↗
40. Wang, S., Li, B.Z., Khabsa, M., Fang, H., and Ma, H. (2020). Linformer: Self-attention with linear complexity. arXiv.
41. Quantum image scaling based on bilinear interpolation with decimals scaling ratio10.1007/s10773-021-04829-6 ↗
42. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images10.1109/TMI.2016.2525803 ↗
43. Gland segmentation in colon histology images: The glas challenge contest10.1016/j.media.2016.08.008 ↗
44. Phoulady, H.A., and Mouton, P.R. (2018). A New Cervical Cytology Dataset for Nucleus Detection and Image Classification (Cervix93) and Methods for Cervical Nucleus Detection. arXiv.
45. Gao, Y., Zhou, M., and Metaxas, D.N. (October, January 27). UTNet: A hybrid transformer architecture for medical image segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France. Proceedings, Part III 24.10.1007/978-3-030-87199-4_6 ↗
46. Huff, D.T., Weisman, A.J., and Jeraj, R. (2021). Interpretation and visualization techniques for deep learning models in medical imaging. Phys. Med. Biol., 66.10.1088/1361-6560/abcd17 ↗