journal article·2023-09Open access

Deep‐MSIM: Fast Image Reconstruction with Deep Learning in Multifocal Structured Illumination Microscopy

Jianhui Liao, Chenshuang Zhang, Xiangcong Xu, Liangliang Zhou, Bin Yu, Danying Lin, Jia Li, Junle Qu

Advanced Science· Volume 10· Issue 27

Abstract

Abstract Fast and precise reconstruction algorithm is desired for for multifocal structured illumination microscopy (MSIM) to obtain the super‐resolution image. This work proposes a deep convolutional neural network (CNN) to learn a direct mapping from raw MSIM images to super‐resolution image, which takes advantage of the computational advances of deep learning to accelerate the reconstruction. The method is validated on diverse biological structures and in vivo imaging of zebrafish at a depth of 100 µm. The results show that high‐quality, super‐resolution images can be reconstructed in one‐third of the runtime consumed by conventional MSIM method, without compromising spatial resolution. Last but not least, a fourfold reduction in the number of raw images required for reconstruction is achieved by using the same network architecture, yet with different training data.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Fluorescence Microscopy Techniques · Image Processing Techniques and Applications · Digital Holography and Microscopy

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

State Key Laboratory of Radio Frequency Heterogeneous Integration Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province College of Physics and Optoelectronic Engineering Shenzhen University Shenzhen 518060 China

Funding

National Key Research and Development Program of China · National Natural Science Foundation of China

References (52)

Show source-provided references
1. 10.1038/nmeth.344210.1038/nmeth.3442 ↗
2. 10.1021/acs.chemrev.6b0072910.1021/acs.chemrev.6b00729 ↗
3. 10.1038/nmeth92910.1038/nmeth929 ↗
4. 10.1364/OL.19.00078010.1364/OL.19.000780 ↗
5. 10.1038/nmeth.459310.1038/nmeth.4593 ↗
6. 10.1046/j.1365-2818.2000.00710.x10.1046/j.1365-2818.2000.00710.x ↗
7. 10.1126/science.aab350010.1126/science.aab3500 ↗
8. 10.1038/nmeth.202510.1038/nmeth.2025 ↗
9. 10.1038/nmeth.268710.1038/nmeth.2687 ↗
10. 10.1073/pnas.131444711110.1073/pnas.1314447111 ↗
11. 10.1088/2050-6120/3/1/01400210.1088/2050-6120/3/1/014002 ↗
12. 10.1364/OE.26.03143010.1364/OE.26.031430 ↗
13. 10.1038/s41598-019-49762-110.1038/s41598-019-49762-1 ↗
14. 10.1364/OE.39083110.1364/OE.390831 ↗
15. 10.1364/OE.41248910.1364/OE.412489 ↗
16. Patterson J. · 2017
17. 10.1038/nature1453910.1038/nature14539 ↗
18. 10.1038/s41592-019-0458-z10.1038/s41592-019-0458-z ↗
19. He K. · 2016 · 770
20. 10.1109/TIP.2017.266220610.1109/TIP.2017.2662206 ↗
21. 10.1364/OPTICA.4.00143710.1364/OPTICA.4.001437 ↗
22. 10.1016/j.cell.2018.02.01010.1016/j.cell.2018.02.010 ↗
23. 10.1364/OPTICA.5.00045810.1364/OPTICA.5.000458 ↗
24. 10.1142/S179354582230016610.1142/S1793545822300166 ↗
25. 10.1364/OE.39235810.1364/OE.392358 ↗
26. 10.1038/s41592-021-01236-x10.1038/s41592-021-01236-x ↗
27. 10.1038/s41592-018-0239-010.1038/s41592-018-0239-0 ↗
28. 10.1186/s43074-022-00077-x10.1186/s43074-022-00077-x ↗
29. 10.1038/s41467-020-15784-x10.1038/s41467-020-15784-x ↗
30. 10.1038/s41592-020-01048-510.1038/s41592-020-01048-5 ↗
31. 10.1038/s41587‐022‐01651‐110.1038/s41587‐022‐01651‐1 ↗
32. 10.1364/OE.45042910.1364/OE.450429 ↗
33. 10.1038/nmeth.201910.1038/nmeth.2019 ↗
34. 10.1038/nmeth.460510.1038/nmeth.4605 ↗
35. 10.1038/nbt.410610.1038/nbt.4106 ↗
36. O.Ronneberger P.Fischer T.Brox inthe 18th International Conference on Medical Image Computing and Computer‐Assisted Intervention (MICCAI) Springer Munich2015 pp. 234–241.10.1007/978-3-319-24574-4_28 ↗
37. Ledig C. · 2017 · 4681
38. Lim B. · 2017 · 1132
39. 10.1109/TPAMI.2015.243928110.1109/TPAMI.2015.2439281 ↗
40. 10.1109/TIP.2003.81986110.1109/TIP.2003.819861 ↗
41. 10.1038/s41592-018-0216-710.1038/s41592-018-0216-7 ↗
42. 10.1038/s41467-020-17546-110.1038/s41467-020-17546-1 ↗
43. 10.1016/j.neunet.2019.08.02510.1016/j.neunet.2019.08.025 ↗
44. Ioffe S. · 2015
45. Glorot X. · 2011
46. 10.23915/distill.0000310.23915/distill.00003 ↗
47. Huang G. · 2017 · 2261
48. Wang Z. · 2003 · 1398
49. 10.1109/TCI.2016.264486510.1109/TCI.2016.2644865 ↗
50. 10.1038/nphoton.2012.8310.1038/nphoton.2012.83 ↗
51. D. P.Kingma J.Ba inthe 3rd International Conference for Learning Representations (ICLR) IEEE San Diego2015.
52. Paszke A. · 2017