journal article·2023-03-14

Soft Robotic Perception System with Ultrasonic Auto-Positioning and Multimodal Sensory Intelligence

Qiongfeng Shi, Zhongda Sun, Xianhao Le, Jin Xie, Chengkuo Lee

ACS Nano· Volume 17· Issue 5
Publisher page

Abstract

Flexible electronics such as tactile cognitive sensors have been broadly adopted in soft robotic manipulators to enable human-skin-mimetic perception. To achieve appropriate positioning for randomly distributed objects, an integrated guiding system is inevitable. Yet the conventional guiding system based on cameras or optical sensors exhibits limited environment adaptability, high data complexity, and low cost effectiveness. Herein, a soft robotic perception system with remote object positioning and a multimodal cognition capability is developed by integrating an ultrasonic sensor with flexible triboelectric sensors. The ultrasonic sensor is able to detect the object shape and distance by reflected ultrasound. Thereby the robotic manipulator can be positioned to an appropriate position to perform object grasping, during which the ultrasonic and triboelectric sensors can capture multimodal sensory information such as object top profile, size, shape, hardness, material, etc. These multimodal data are then fused for deep-learning analytics, leading to a highly enhanced accuracy in object identification (∼100%). The proposed perception system presents a facile, low-cost, and effective methodology to integrate positioning capability with multimodal cognitive intelligence in soft robotics, significantly expanding the functionalities and adaptabilities of current soft robotic systems in industrial, commercial, and consumer applications.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Advanced Sensor and Energy Harvesting Materials · Tactile and Sensory Interactions · Soft Robotics and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Joint International Research Laboratory of Information Display and Visualization, School of Electronic Science and Engineering, Southeast University, Nanjing 210096, China

Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583, Singapore

Center for Intelligent Sensors and MEMS (CISM), National University of Singapore, Singapore 117608, Singapore

National University of Singapore Suzhou Research Institute (NUSRI), Suzhou Industrial Park, Suzhou 215123, China

State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China

NUS Graduate School - Integrative Sciences and Engineering Programme (ISEP), National University of Singapore, Singapore 119077, Singapore

Funding

Ministry of Science and Technology of the People's Republic of China · National University of Singapore · National Natural Science Foundation of China · Institute for Health Innovation & Technology, National University of Singapore · SIMTech ? National University of Singapore Joint Laboratory for Large-Area Flexible Hybrid Electronics

References (68)

Show source-provided references
1. 10.1126/scirobotics.aau847910.1126/scirobotics.aau8479 ↗
2. 10.1016/j.procs.2018.07.02510.1016/j.procs.2018.07.025 ↗
3. 10.1089/soro.2017.000710.1089/soro.2017.0007 ↗
4. 10.1126/scirobotics.aaz923910.1126/scirobotics.aaz9239 ↗
5. 10.1002/advs.20180054110.1002/advs.201800541 ↗
6. 10.1126/scirobotics.aav148810.1126/scirobotics.aav1488 ↗
7. 10.3390/s2119634010.3390/s21196340 ↗
8. 10.1016/j.jmsy.2020.07.01310.1016/j.jmsy.2020.07.013 ↗
9. 10.1038/s41586-019-1479-610.1038/s41586-019-1479-6 ↗
10. 10.1126/sciadv.aax574610.1126/sciadv.aax5746 ↗
11. 10.1126/scirobotics.aan426810.1126/scirobotics.aan4268 ↗
12. 10.1126/scirobotics.aay360410.1126/scirobotics.aay3604 ↗
13. 10.1126/scirobotics.aav148810.1126/scirobotics.aav1488 ↗
14. 10.1126/science.aba550410.1126/science.aba5504 ↗
15. 10.1016/j.sna.2015.09.04110.1016/j.sna.2015.09.041 ↗
16. 10.1126/scirobotics.aai752910.1126/scirobotics.aai7529 ↗
17. 10.1002/aisy.20190001810.1002/aisy.201900018 ↗
18. 10.1063/1.507418410.1063/1.5074184 ↗
19. 10.1002/adma.20180240510.1002/adma.201802405 ↗
20. 10.1021/nl300303910.1021/nl3003039 ↗
21. 10.1016/j.nanoen.2012.01.00410.1016/j.nanoen.2012.01.004 ↗
22. 10.1016/j.mattod.2019.05.01610.1016/j.mattod.2019.05.016 ↗
23. 10.1021/acsnano.1c0757910.1021/acsnano.1c07579 ↗
24. 10.1038/s41467-022-32745-810.1038/s41467-022-32745-8 ↗
25. 10.1016/j.nanoen.2019.10427210.1016/j.nanoen.2019.104272 ↗
26. 10.1002/aenm.20180290610.1002/aenm.201802906 ↗
27. 10.1002/eom2.1205810.1002/eom2.12058 ↗
28. 10.1016/j.nanoen.2019.01.00210.1016/j.nanoen.2019.01.002 ↗
29. 10.1016/j.nanoen.2020.10530710.1016/j.nanoen.2020.105307 ↗
30. 10.1016/j.nanoen.2022.10742810.1016/j.nanoen.2022.107428 ↗
31. 10.1002/aisy.20210022810.1002/aisy.202100228 ↗
32. 10.1021/acs.analchem.2c0452710.1021/acs.analchem.2c04527 ↗
33. 10.1016/j.nanoen.2020.10477210.1016/j.nanoen.2020.104772 ↗
34. 10.1021/acsnano.8b0256210.1021/acsnano.8b02562 ↗
35. 10.1002/adfm.20190731210.1002/adfm.201907312 ↗
36. 10.1016/j.nanoen.2022.10713710.1016/j.nanoen.2022.107137 ↗
37. 10.1002/adfm.20220827110.1002/adfm.202208271 ↗
38. 10.1021/acsnano.2c0404310.1021/acsnano.2c04043 ↗
39. 10.1002/aenm.20220304010.1002/aenm.202203040 ↗
40. 10.1002/advs.20220105610.1002/advs.202201056 ↗
41. 10.1021/acsmaterialsau.2c0000110.1021/acsmaterialsau.2c00001 ↗
42. 10.1038/s41467-020-18471-z10.1038/s41467-020-18471-z ↗
43. 10.1038/s41467-020-19059-310.1038/s41467-020-19059-3 ↗
44. 10.1002/advs.20210023010.1002/advs.202100230 ↗
45. 10.1109/TRO.2019.290446110.1109/TRO.2019.2904461 ↗
46. 10.1016/j.patcog.2018.04.01710.1016/j.patcog.2018.04.017 ↗
47. 10.1038/s41551-018-0287-x10.1038/s41551-018-0287-x ↗
48. 10.1038/s41551-021-00685-110.1038/s41551-021-00685-1 ↗
49. Zhu X. · 2019 · 2 · 2069
50. 10.1126/sciadv.abg250710.1126/sciadv.abg2507 ↗
51. 10.1126/sciadv.abi928310.1126/sciadv.abi9283 ↗
52. 10.3390/mi1101000710.3390/mi11010007 ↗
53. 10.1088/1361-6439/ac3ab910.1088/1361-6439/ac3ab9 ↗
54. 10.1088/1361-6439/aa851b10.1088/1361-6439/aa851b ↗
55. 10.1109/JMEMS.2015.247295810.1109/JMEMS.2015.2472958 ↗
56. 10.1109/LED.2015.245907510.1109/LED.2015.2459075 ↗
57. 10.1038/srep2494610.1038/srep24946 ↗
58. 10.1063/1.494897310.1063/1.4948973 ↗
59. Liu, X.; Chen, X.; Le, X.; Xu, Z.; Wu, C.; Xie, J. A High-Performance Square PMUT for Range-Finder. In 2018 IEEE 13th Annual International Conference on Nano/Micro Engineered and Molecular Systems (NEMS); IEEE, 2018; pp 106–109.10.1109/NEMS.2018.8556913 ↗
60. 10.1109/TIE.2018.287458310.1109/TIE.2018.2874583 ↗
61. 10.1016/j.nanoen.2019.01.09610.1016/j.nanoen.2019.01.096 ↗
62. 10.1063/1.492291510.1063/1.4922915 ↗
63. 10.1109/JMEMS.2015.250333610.1109/JMEMS.2015.2503336 ↗
64. 10.1109/JMEMS.2019.291286910.1109/JMEMS.2019.2912869 ↗
65. 10.3390/s1921469610.3390/s19214696 ↗
66. 10.1109/TUFFC.2005.156327010.1109/TUFFC.2005.1563270 ↗
67. 10.1002/advs.20200026110.1002/advs.202000261 ↗
68. 10.1038/s41928-020-0422-z10.1038/s41928-020-0422-z ↗