Unsupervised Segmentation of 3D Microvascular Photoacoustic Images Using Deep Generative Learning
Source abstract
Abstract
Abstract Mesoscopic photoacoustic imaging (PAI) enables label‐free visualization of vascular networks in tissues with high contrast and resolution. Segmenting these networks from 3D PAI data and interpreting their physiological and pathological significance is crucial yet challenging due to the time‐consuming and error‐prone nature of current methods. Deep learning offers a potential solution; however, supervised analysis frameworks typically require human‐annotated ground‐truth labels. To address this, an unsupervised image‐to‐image translation deep learning model is introduced, the Vessel Segmentation Generative Adversarial Network (VAN‐GAN). VAN‐GAN integrates synthetic blood vessel networks that closely resemble real‐life anatomy into its training process and learns to replicate the underlying physics of the PAI system in order to learn how to segment vasculature from 3D photoacoustic images. Applied to a diverse range of in silico, in vitro, and in vivo data, including patient‐derived breast cancer xenograft models and 3D clinical angiograms, VAN‐GAN demonstrates its capability to facilitate accurate and unbiased segmentation of 3D vascular networks. By leveraging synthetic data, VAN‐GAN reduces the reliance on manual labeling, thus lowering the barrier to entry for high‐quality blood vessel segmentation (F1 score: VAN‐GAN vs. U‐Net = 0.84 vs. 0.87) and enhancing preclinical and clinical research into vascular structure and function.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Photoacoustic and Ultrasonic Imaging · Image Enhancement Techniques · Optical Imaging and Spectroscopy Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Cancer Research UK Cambridge Institute University of Cambridge Robinson Way Cambridge CB2 0RE UK
Department of Physics University of Cambridge JJ Thomson Avenue Cambridge CB3 0HE UK
Funding
Cancer Research UK · Wellcome Trust · Cambridge Trust · National Physical Laboratory · Engineering and Physical Sciences Research Council
References (86)
Show source-provided references
1. 10.1021/acsphotonics.2c0126410.1021/acsphotonics.2c01264 ↗
2. 10.1038/s41377-022-00991-610.1038/s41377-022-00991-6 ↗
3. 10.1016/j.csbj.2020.08.00310.1016/j.csbj.2020.08.003 ↗
4. Ronneberger O. · 2015
5. 10.1007/978-3-319-46723-8_4910.1007/978-3-319-46723-8_49 ↗
6. F.Milletari N.Navab S. A.Ahmadi inProceedings ‐ 2016 4th International Conference on 3D Vision Institute of Electrical and Electronics Engineers Inc. Stanford CA USA2016 pp.565–571.10.1109/3DV.2016.79 ↗
7. 10.1038/s41551-018-0306-y10.1038/s41551-018-0306-y ↗
8. 10.1016/j.mvr.2010.03.00710.1016/j.mvr.2010.03.007 ↗
9. 10.1109/TMI.2004.82562710.1109/TMI.2004.825627 ↗
10. 10.1109/42.84517810.1109/42.845178 ↗
11. P.Teikari M.Santos C.Poon K.Hynynen arXiv:1606.023822016.
12. 10.1371/journal.pone.021353910.1371/journal.pone.0213539 ↗
13. M.Haft‐Javaherian M.Villiger C. B.Schaffer N.Nishimura P.Golland B. E.Bouma inIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops vol.2020 IEEE Piscataway NJ2020 pp.4262–4271.
14. 10.3390/sym1207106710.3390/sym12071067 ↗
15. 10.1364/BOE.45800410.1364/BOE.458004 ↗
16. 10.3390/jimaging810025910.3390/jimaging8100259 ↗
17. 10.1109/TMI.2022.319315010.1109/TMI.2022.3193150 ↗
18. F.Lin Y.Xia N.Ravikumar Q.Liu M.MacRaild A. F.Frangi arXiv:2308.036132023.
19. 10.1109/TMM.2021.310941910.1109/TMM.2021.3109419 ↗
20. A. K.Mondal A.Agarwal J.Dolz C.Desrosiers arXiv:1908.115692019.
21. 10.1038/s42256-019-0096-210.1038/s42256-019-0096-2 ↗
22. 10.1109/TMI.2019.292718210.1109/TMI.2019.2927182 ↗
23. 10.1016/j.media.2021.10200410.1016/j.media.2021.102004 ↗
24. R.Chen T. T.Yu G.Xu D.Ma M. V.Sarunic M. F.Beg arXiv:2107.023452021.
25. C.Fu S.Lee D. J.Ho S.Han P.Salama K. W.Dunn E. J.Delp inIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops vol.2018 IEEE Piscataway NJ2018 pp.2302–2310.10.1109/CVPRW.2018.00298 ↗
26. Y.Ma Y.Hua H.Deng T.Song H.Wang Z.Xue H.Cao R.Ma H.Guan inProceedings of the IEEE International Conference on Computer Vision.Institute of Electrical and Electronics Engineers (IEEE) IEEE Piscataway NJ2021 pp.7516–7525.10.1109/ICCV48922.2021.00744 ↗
27. J.Gröhl K. K.Dreher M.Schellenberg A.Seitel L.Maier‐Hein Photons Plus Ultrasound Imaging Sens 2021 11642 116423C.
28. Lan H. · 2019 · 273
29. 10.1002/jbio.20200032510.1002/jbio.202000325 ↗
30. C.Bench B. T.Cox arXiv:2305.047142023.
31. 10.1109/TMI.2023.333119810.1109/TMI.2023.3331198 ↗
32. 10.1364/BOE.40924610.1364/BOE.409246 ↗
33. 10.1016/j.pacs.2022.10034110.1016/j.pacs.2022.100341 ↗
34. 10.1016/j.pacs.2021.10031010.1016/j.pacs.2021.100310 ↗
35. 10.1109/TMI.2022.315847410.1109/TMI.2022.3158474 ↗
36. 10.1038/s41551-023-01151-w10.1038/s41551-023-01151-w ↗
37. He H. · IEEE Trans. Med. Imaging · 2024
38. 10.1016/j.pacs.2022.10035710.1016/j.pacs.2022.100357 ↗
39. 10.1016/j.pacs.2023.10050510.1016/j.pacs.2023.100505 ↗
40. S.Gur L.Wolf L.Golgher P.Blinder inProceedings of the IEEE International Conference on Computer Vision vol.2019 IEEE Piscataway NJ2019 pp.10721–10730.10.1109/ICCV.2019.01082 ↗
41. Cui H. · Int. J. Comput. Intell. Syst. · 2021 · 14 · 1
42. J. Y.Zhu T.Park P.Isola A. A.Efros inProceedings of the IEEE International Conference on Computer Vision IEEE Piscataway NJ2017 pp.2242–2251.10.1109/ICCV.2017.244 ↗
43. K.He X.Zhang S.Ren J.Sun inProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE Computer Society Piscataway NJ2016 pp.770–778.10.1109/CVPR.2016.90 ↗
44. P.Isola J. Y.Zhu T.Zhou A. A.Efros Proceedings ‐ 30th IEEE Conference on Computer Vision and Pattern Recognition CVPR 2017 IEEE Piscataway NJ2017 pp.5967–5976.10.1109/CVPR.2017.632 ↗
45. S.Shit J. C.Paetzold A.Sekuboyina I.Ezhov A.Unger A.Zhylka J. P. W.Pluim U.Bauer B. H.Menze inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition IEEE Piscataway NJ2021 pp.16560–16569.
46. C. K.Sønderby J.Caballero L.Theis W.Shi F.Huszár in5th International Conference on Learning Representations (ICLR) Toulon France 2017 Conference Track Proceedings.
47. M.Arjovsky L.Bottou in5th International Conference on Learning Representations (ICLR) Toulon France 2017 Conference Track Proceedings.
48. 10.1038/s41592-019-0582-910.1038/s41592-019-0582-9 ↗
49. 10.1158/1078-0432.CCR-16-320010.1158/1078-0432.CCR-16-3200 ↗
50. 10.1038/s41551-020-0527-810.1038/s41551-020-0527-8 ↗
51. 10.1016/j.cell.2016.08.04110.1016/j.cell.2016.08.041 ↗
52. Lin L. · Nat. Rev. Clin. Oncol. · 2022 · 1
53. 10.1016/j.pacs.2023.10053910.1016/j.pacs.2023.100539 ↗
54. Seno Medical Instruments Inc. Imagio Breast Imaging System ‐ P200003 2021 FDA Premarket Approval.
55. 10.1038/s41467-017-02088-w10.1038/s41467-017-02088-w ↗
56. 10.1038/s41467-020-20314-w10.1038/s41467-020-20314-w ↗
57. 10.1117/1.JBO.29.S1.S1150610.1117/1.JBO.29.S1.S11506 ↗
58. M.Abadi A.Agarwal P.Barham E.Brevdo Z.Chen C.Citro G. S.Corrado A.Davis J.Dean M.Devin S.Ghemawat I.Goodfellow A.Harp G.Irving M.Isard Y.Jia R.Jozefowicz L.Kaiser M.Kudlur J.Levenberg D.Mané M.Schuster R.Monga S.Moore D.Murray C.Olah M.Schuster J.Shlens B.Steiner I.Sutskever et al. in12th USENIX Symposium on Operating Systems Design and Implementation USENIX Association Berkeley CA2015 pp.265–278.
59. F.Chollet Others Keras https://keras.io(accessed: 2015).
60. J.Montalt Tordera TensorFlow MRI https://zenodo.org/record/5752336(accessed: 2021).
61. 10.1109/LGRS.2018.280294410.1109/LGRS.2018.2802944 ↗
62. J.Long E.Shelhamer T.Darrell 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IEEE Piscataway NJ2015 pp.3431–3440.
63. D.Ulyanov A.Vedaldi V.Lempitsky arXiv:1607.080222016.
64. V.Nair G. E.Hinton inICML 2010 ‐ Proceedings 27th International Conference on Machine Learning Association for Computing Machinery New York2010 pp.807–814.
65. B.Alsallakh N.Kokhlikyan F.Ai V.Miglani J.Yuan O.Reblitz‐Richardson in9th International Conference on Learning Representations (ICLR) Vienna Austria2021.
66. A. L.Maas A. Y.Hannun A. Y.Ng inICML Workshop on Deep Learning for Audio Speech and Language Processing.2013.
67. J.Tompson R.Goroshin A.Jain Y.LeCun C.Bregler inProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society IEEE Piscataway NJ2014 pp.648–656.
68. 10.1109/TCI.2016.264486510.1109/TCI.2016.2644865 ↗
69. 10.1186/1687-6180-2012-1610.1186/1687-6180-2012-16 ↗
70. Nay D. J. B. · SSRN Electron. J. · 2021
71. 10.1016/j.patcog.2021.10835110.1016/j.patcog.2021.108351 ↗
72. X.Mao Q.Li H.Xie R. Y.Lau Z.Wang S. P.Smolley inIEEE International Conference on Computer Vision (ICCV) IEEE Venice Italy 2017 pp.2813–2821.10.1109/ICCV.2017.304 ↗
73. 10.1016/0022-5193(68)90079-910.1016/0022-5193(68)90079-9 ↗
74. M. A.Galarreta‐Valverde Ph.D. thesis Instituto de Matemática e Estatística 2012.
75. 10.1117/12.200753210.1117/12.2007532 ↗
76. 10.1117/1.JBO.27.8.08301010.1117/1.JBO.27.8.083010 ↗
77. 10.1117/1.336030810.1117/1.3360308 ↗
78. 10.1038/s41416-018-0033-x10.1038/s41416-018-0033-x ↗
79. 10.2967/jnumed.116.18231110.2967/jnumed.116.182311 ↗
80. D. P.Kingma J. L.Ba 3rd International Conference on Learning Representations ICLR 2015 ‐ Conference Track ProceedingsSan Diego USA2014.
81. R.Pascanu T.Mikolov Y.Bengio in30th International Conference on Machine Learning ICML 2013 Atlanta USA2013 pp.2347–2355.
82. 10.1038/s41592-023-02151-z10.1038/s41592-023-02151-z ↗
83. 10.1126/sciadv.abm245610.1126/sciadv.abm2456 ↗
84. R.Bates Ph.D. thesis University of Oxford 2017.
85. R.Bates Russ‐learn: set of tools for application and training of deep learning methods for image segmentation and vessel analysis https://ibme‐gitcvs.eng.ox.ac.uk/RussellB/unet‐test(accessed: 2018).
86. 10.1162/netn_a_0009110.1162/netn_a_00091 ↗