journal article·2025-07Open access

Skin‐Conformal Motion Monitoring Film for Deep‐Learning‐Based Immersive Extended Reality

Jihyeon Ahn, Taehwan Kim, Ji‐Hwan Ha, Donho Lee, Osman Gul, Seokjoo Cho, Hyunjin Kim, Mingu Kang, Jungrak Choi, Junseong Ahn, Inkyu Park

Advanced Functional Materials· Volume 35· Issue 28

Abstract

Abstract Immersive technologies like virtual reality (VR) and augmented reality (AR) are rapidly advancing, but current motion tracking systems face significant limitations. Camera‐based methods struggle with occlusion and environmental sensitivity, while existing skin‐applied sensors suffer from poor adhesion or lack of reusability. This research introduces a deep‐learning‐based motion monitoring system utilizing a skin‐conformal stretchable film for reliable and immersive motion tracking in VR and AR applications. The approach enhances skin adhesion through elastomer cross‐linking weakening and microwrinkle formation. The developed skin‐conformal motion monitoring film (SCMF) demonstrates 5.3 times higher skin adhesion (0.0218 N mm−1) compared to conventional elastomer. With its improved adhesion, the SCMF demonstrates high conformality to skin movements, accurately following dynamic motions of the skin, resulting in reliable strain and bending angle signals. By applying the SCMF to finger joints, a system capable of reliably monitoring finger movements in a VR environment and classifying baseball grips using convolutional neural network (CNN) algorithms can be developed. The application is extended to 7 upper body parts, developing a real‐time workout monitoring system in AR using change point detection and 2D CNN algorithms. The approach enhances accuracy and adaptability of motion tracking across immersive technologies, potentially transforming user experiences in various sectors.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Tactile and Sensory Interactions · Virtual Reality Applications and Impacts · Advanced Sensor and Energy Harvesting Materials

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Mechanical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141 Republic of Korea

Center for Nano Lithography & Manufacturing Korea Institute of Machinery and Materials (KIMM) Daejeon 34103 Republic of Korea

Electronics and Telecommunications Research Institute (ETRI) Daejeon 34129 Republic of Korea

Department of Control and Instrumentation Engineering Korea University Sejong 30019 Republic of Korea

Funding

Korea Evaluation Institute of Industrial Technology · National Research Foundation

References (64)

Show source-provided references
1. 10.1038/s41586-024-07386-010.1038/s41586-024-07386-0 ↗
2. 10.1126/science.adk375510.1126/science.adk3755 ↗
3. 10.1038/s41586-024-07382-410.1038/s41586-024-07382-4 ↗
4. Zhu M. · Sci. Adv. · 2020 · 6 · aaz869
5. 10.1038/s41467-022-33716-910.1038/s41467-022-33716-9 ↗
6. 10.1038/s41467-023-37678-410.1038/s41467-023-37678-4 ↗
7. 10.1002/advs.20230323410.1002/advs.202303234 ↗
8. 10.1016/j.nanoen.2023.10890310.1016/j.nanoen.2023.108903 ↗
9. 10.1038/s41467-024-51516-110.1038/s41467-024-51516-1 ↗
10. 10.1038/s41587-019-0045-y10.1038/s41587-019-0045-y ↗
11. 10.1002/advs.20200026110.1002/advs.202000261 ↗
12. Ji B. · Int. J. Hum. Comput. Interact. · 2023 · 39 · 6793
13. 10.1038/s41928-023-00968-210.1038/s41928-023-00968-2 ↗
14. 10.1038/s41467-020-16040-y10.1038/s41467-020-16040-y ↗
15. 10.1038/nnano.2017.12510.1038/nnano.2017.125 ↗
16. 10.1038/s41928-023-01012-z10.1038/s41928-023-01012-z ↗
17. 10.1002/adfm.20200749510.1002/adfm.202007495 ↗
18. 10.1021/acsami.9b1806910.1021/acsami.9b18069 ↗
19. 10.1002/adfm.20230356210.1002/adfm.202303562 ↗
20. 10.1002/admt.20230037810.1002/admt.202300378 ↗
21. 10.1002/smll.20230361210.1002/smll.202303612 ↗
22. 10.1002/aenm.20220381210.1002/aenm.202203812 ↗
23. 10.1016/j.nanoen.2024.11004510.1016/j.nanoen.2024.110045 ↗
24. 10.1016/j.nanoen.2021.10644710.1016/j.nanoen.2021.106447 ↗
25. 10.1002/smll.20230398110.1002/smll.202303981 ↗
26. 10.1002/adfm.20220879210.1002/adfm.202208792 ↗
27. 10.1016/j.cej.2024.15495310.1016/j.cej.2024.154953 ↗
28. 10.1039/D2MA00897A10.1039/D2MA00897A ↗
29. 10.1039/D2NJ03297J10.1039/D2NJ03297J ↗
30. 10.1002/mame.20220003410.1002/mame.202200034 ↗
31. 10.1002/adhm.20200146110.1002/adhm.202001461 ↗
32. 10.1002/adma.20231050510.1002/adma.202310505 ↗
33. 10.1038/s41586-019-1687-010.1038/s41586-019-1687-0 ↗
34. 10.1002/adma.20150537210.1002/adma.201505372 ↗
35. 10.1038/s41427-019-0150-x10.1038/s41427-019-0150-x ↗
36. 10.1021/acsami.2c1265810.1021/acsami.2c12658 ↗
37. 10.1021/acsami.8b1894710.1021/acsami.8b18947 ↗
38. 10.1126/sciadv.adf104310.1126/sciadv.adf1043 ↗
39. 10.1016/j.cej.2021.13246710.1016/j.cej.2021.132467 ↗
40. 10.1152/ajpregu.00178.201810.1152/ajpregu.00178.2018 ↗
41. Hu T. · ACS Appl. Mater. Interfaces · 2024 · 16
42. 10.1038/nnano.2011.3610.1038/nnano.2011.36 ↗
43. Cai L. · Sci. Rep. · 2013 · 3
44. H. N. Hs. O. I. H · J. Sens. · 2017
45. 10.1016/j.aej.2021.04.02110.1016/j.aej.2021.04.021 ↗
46. 10.1039/C4NR03295K10.1039/C4NR03295K ↗
47. R. P.Adams D. J. C.MacKay arXiv2007 10703742.
48. 10.3390/s2001016810.3390/s20010168 ↗
49. Y.Sharma N.Coronato D. E.Brown presented at2022 44th Annual Int. Conf. IEEE E Medicine Biology Society (EMBC) Glasgow Scotland UK July2022.
50. 10.1038/s41467-024-50797-w10.1038/s41467-024-50797-w ↗
51. 10.1038/s41467-023-38274-210.1038/s41467-023-38274-2 ↗
52. 10.1038/s41528-021-00114-y10.1038/s41528-021-00114-y ↗
53. 10.1016/j.nanoen.2021.10597810.1016/j.nanoen.2021.105978 ↗
54. 10.1002/adma.20230087110.1002/adma.202300871 ↗
55. 10.1038/s41467-023-36302-910.1038/s41467-023-36302-9 ↗
56. 10.1021/acsami.7b1772710.1021/acsami.7b17727 ↗
57. 10.1002/adma.20180330910.1002/adma.201803309 ↗
58. 10.1016/j.nanoen.2021.10583710.1016/j.nanoen.2021.105837 ↗
59. 10.1002/advs.20180010010.1002/advs.201800100 ↗
60. 10.1002/adfm.20180522410.1002/adfm.201805224 ↗
61. 10.1002/smll.20240549310.1002/smll.202405493 ↗
62. 10.1038/s41528-023-00238-310.1038/s41528-023-00238-3 ↗
63. 10.1002/adfm.20231619610.1002/adfm.202316196 ↗
64. 10.1038/s41467-021-25324-w10.1038/s41467-021-25324-w ↗