journal article·2023-12-20Open access

Achromatic Imaging Systems with Flat Lenses Enabled by Deep Learning

Roy Maman, Eitan Mualem, Noa Mazurski, Jacob Engelberg, Uriel Levy

ACS Photonics· Volume 10· Issue 12

Abstract

High Resolution Image Download MS PowerPoint Slide Motivated by their great potential to reduce the size, cost, and weight, flat lenses, a category that includes diffractive lenses and metalenses, are rapidly emerging as key components with the potential to replace the traditional refractive optical elements in modern optical systems. Yet, the inherently strong chromatic aberration of these flat lenses is significantly impairing their performance in systems based on polychromatic illumination or passive ambient light illumination, stalling their widespread implementation. Hereby, we provide a promising solution and demonstrate high-quality imaging based on flat lenses over the entire visible spectrum. Our approach is based on creating a novel data set of color outdoor images taken with our flat lens and using this data set to train a deep-learning model for chromatic aberrations correction. Based on this approach, we show unprecedented imaging results not only in terms of qualitative measures but also in quantitative terms of the peak signal-to-noise ratio (PSNR) and structure similarity index (SSIM) scores of the reconstructed images. The results pave the way for the implementation of flat lenses in advanced polychromatic imaging systems.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Advanced optical system design · Optical Coatings and Gratings · Adaptive optics and wavefront sensing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Institute of Applied Physics, The Faculty of Science, The Center for Nanoscience and Nanotechnology, The Hebrew University of Jerusalem, Jerusalem 91904, Israel

Funding

Israel Innovation Authority

References (68)

Show source-provided references
1. 10.1038/s41566-022-00963-710.1038/s41566-022-00963-7 ↗
2. 10.1364/JOSAA.23.00056710.1364/JOSAA.23.000567 ↗
3. 10.1016/0030-4018(95)00071-F10.1016/0030-4018(95)00071-F ↗
4. 10.1364/AO.28.00395010.1364/AO.28.003950 ↗
5. 10.1364/OPTICA.6.00080510.1364/OPTICA.6.000805 ↗
6. 10.1038/s41467-020-15972-910.1038/s41467-020-15972-9 ↗
7. 10.1088/1464-4258/4/5/35810.1088/1464-4258/4/5/358 ↗
8. 10.1038/ncomms1368210.1038/ncomms13682 ↗
9. 10.1126/science.aaf664410.1126/science.aaf6644 ↗
10. 10.1364/OE.22.02621210.1364/OE.22.026212 ↗
11. 10.1364/OL.25.00169010.1364/OL.25.001690 ↗
12. 10.1038/ncomms806910.1038/ncomms8069 ↗
13. 10.1063/1.153930010.1063/1.1539300 ↗
14. 10.1364/OE.25.02163710.1364/OE.25.021637 ↗
15. 10.1126/science.aaa249410.1126/science.aaa2494 ↗
16. 10.1038/s41565-017-0034-610.1038/s41565-017-0034-6 ↗
17. 10.1038/s41377-018-0078-x10.1038/s41377-018-0078-x ↗
18. 10.1364/OPTICA.3.00062810.1364/OPTICA.3.000628 ↗
19. 10.1364/OPTICA.4.00062510.1364/OPTICA.4.000625 ↗
20. 10.1038/s41565-017-0052-410.1038/s41565-017-0052-4 ↗
21. 10.1038/ncomms1499210.1038/ncomms14992 ↗
22. 10.1126/sciadv.aar211410.1126/sciadv.aar2114 ↗
23. 10.1038/s41598-018-21169-410.1038/s41598-018-21169-4 ↗
24. 10.1364/OL.39.00316910.1364/OL.39.003169 ↗
25. 10.1364/AO.34.00246910.1364/AO.34.002469 ↗
26. 10.1364/AO.34.00246210.1364/AO.34.002462 ↗
27. 10.1364/OPTICA.42284310.1364/OPTICA.422843 ↗
28. 10.1364/OPTICA.38940410.1364/OPTICA.389404 ↗
29. 10.1109/TIP.2019.292986510.1109/TIP.2019.2929865 ↗
30. Li, K.; Jin, X. InChromatic Aberration Correction Using Cross-Channel Prior in Shearlet Domain, Proceedings of the Asian Conference on Computer Vision, 2020; pp 102–117.10.1007/978-3-030-69532-3_7 ↗
31. 10.1145/1409060.140906910.1145/1409060.1409069 ↗
32. Michaeli, T.; Irani, M. InNonparametric Blind Super-Resolution, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2013; pp 945–952.10.1109/ICCV.2013.121 ↗
33. Timofte, R.; De Smet, V.; Van Gool, L. InAnchored Neighborhood Regression for Fast Example-Based Super-Resolution, 2013 IEEE International Conference on Computer Vision, 2013; pp 1920–1927.10.1109/ICCV.2013.241 ↗
34. 10.3390/s2301041510.3390/s23010415 ↗
35. Nikonorov, A.; Evdokimova, V.; Petrov, M.; Yakimov, P.; Bibikov, S.; Yuzifovich, Y.; Skidanov, R.; Kazanskiy, N. InDeep Learning-Based Imaging Using Single-Lens and Multi-Aperture Diffractive Optical Systems, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.10.1109/ICCVW.2019.00491 ↗
36. 10.1038/s41467-022-30439-910.1038/s41467-022-30439-9 ↗
37. 10.1038/s41467-021-26443-010.1038/s41467-021-26443-0 ↗
38. Schuler, C. J.; Hirsch, M.; Harmeling, S.; Schölkopf, B. InNon-Stationary Correction of Optical Aberrations, 2011 International Conference on Computer Vision, 2011; pp 659–666.10.1109/ICCV.2011.6126301 ↗
39. He, C.; Hu, C.H.; Zhang, W. InAdaptive Shearlet-Regularized Image Deblurring via Alternating Direction Method, 2014 IEEE International Conference on Multimedia and Expo (ICME); IEEE, 2014; pp 1–6.10.1109/ICME.2014.6890184 ↗
40. 10.1109/TIP.2019.292986510.1109/TIP.2019.2929865 ↗
41. 10.1109/TCSVT.2020.299062310.1109/TCSVT.2020.2990623 ↗
42. 10.1109/TIP.2013.228632810.1109/TIP.2013.2286328 ↗
43. Gokstorp, M. InComputing Depth from Out-of-Focus Blur Using a Local Frequency Representation, Proceedings of 12th International Conference on Pattern Recognition, 1994; pp 153–158.10.1109/ICPR.1994.576248 ↗
44. Joshi, N.; Szeliski, R.; Kriegman, D. J. InPSF Estimation Using Sharp Edge Prediction, 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008; pp 1–8.10.1109/CVPR.2008.4587834 ↗
45. 10.48550/arXiv.1806.0291910.48550/arXiv.1806.02919 ↗
46. 10.1109/TPAMI.2020.296852110.1109/TPAMI.2020.2968521 ↗
47. Zhang, K.; Zuo, W.; Gu, S.; Zhang, L. InLearning Deep CNN Denoiser Prior for Image Restoration, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017; pp 2808–2817.10.1109/CVPR.2017.300 ↗
48. 10.1109/TPAMI.2020.302108810.1109/TPAMI.2020.3021088 ↗
49. Vaswani A. · 2017 · 30
50. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. 2023, arXiv:2010.11929. arXiv.org e-Print archive. http://arxiv.org/abs/2010.11929.
51. Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jegou, H. InTraining Data-Efficient Image Transformers & Distillation through Attention, Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, PMLR; Meila, M.; Zhang, T., Eds. 2021; pp 10347–10357.
52. Yuan, L.; Chen, Y.; Wang, T.; Yu, W.; Shi, Y.; Jiang, Z. H.; Tay, F. E.; Feng, J.; Yan, S. InTokens-to-Token Vit: Training Vision Transformers from Scratch on Imagenet, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021; pp 558–567.10.1109/ICCV48922.2021.00060 ↗
53. Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. InSwin Transformer: Hierarchical Vision Transformer Using Shifted Windows, 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021; pp 9992–10002.10.1109/ICCV48922.2021.00986 ↗
54. Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; Zagoruyko, S. InEnd-to-End Object Detection with Transformers, European Conference on Computer Vision; Springer International Publishing: Cham, 2020; pp 213–229.10.1007/978-3-030-58452-8_13 ↗
55. 10.1145/350524410.1145/3505244 ↗
56. Zamir, S. W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F. S.; Yang, M. H. InRestormer: Efficient Transformer for High-Resolution Image Restoration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022; pp 5728–5739.10.1109/CVPR52688.2022.00564 ↗
57. Li, Y.; Fan, Y.; Xiang, X.; Demandolx, D.; Ranjan, R.; Timofte, R.; Van Gool, L. InEfficient and Explicit Modelling of Image Hierarchies for Image Restoration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023; pp 18278–18289.10.1109/CVPR52729.2023.01753 ↗
58. Zhang, D.; Wang, X. InDynamic Multi-Scale Network for Dual-Pixel Images Defocus Deblurring with Transformer, 2022 IEEE International Conference on Multimedia and Expo (ICME); IEEE, 2022; pp 1–6.10.1109/ICME52920.2022.9859631 ↗
59. Kumar, M.; Weissenborn, D.; Kalchbrenner, N. Colorization Transformer. 2021, arXiv:2102.04432. arXiv.org e-Print archive. http://arxiv.org/abs/2102.04432.
60. 10.1016/j.patcog.2021.10786110.1016/j.patcog.2021.107861 ↗
61. Zach C. · 2007 · 21410.1007/978-3-540-74936-3_22 ↗
62. Wedel A. · 2009 · 2310.1007/978-3-642-03061-1_2 ↗
63. 10.5201/ipol.2013.2610.5201/ipol.2013.26 ↗
64. Abuolaim A. · 2020 · 11110.1007/978-3-030-58607-2_7 ↗
65. Loshchilov, I.; Hutter, F. Sgdr: Stochastic Gradient Descent with Warm Restarts. 2023, arXiv:1608.03983. arXiv.org e-Print archive. http://arxiv.org/abs/1608.03983.
66. 10.1109/TIP.2003.81986110.1109/TIP.2003.819861 ↗
67. Purohit, K.; Suin, M.; Rajagopalan, A. N.; Boddeti, V. N. InSpatially-Adaptive Image Restoration Using Distortion-Guided Networks, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021; pp 2309–2319.10.1109/ICCV48922.2021.00231 ↗
68. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional Networks for Biomedical Image Segmentation. InMedical Image Computing and Computer-Assisted Intervention─MICCAI 2015, 18th International Conference, Part III, Munich, Germany, 2015; pp 234–241.10.1007/978-3-319-24574-4_28 ↗