Design and evaluation of touch and touchless interfaces for interactive digital heritage experiences
Source abstract
Abstract
With the development of Information and Communication Technology (ICT), cultural heritage has adopted new forms of display by integrating digital and physical realms. Interaction methods have expanded from traditional touchscreen input to touchless input, enriching user engagement. However, the differences between touch and touchless interaction modes -and their impact on experience remain insufficiently explored. This paper presents a virtual museum themed around blue-and-white porcelain. We systematically designed gestures, interfaces, and tasks for user to experience and evaluate. Participants were assigned three tasks-translation, rotation, and scaling. Both objective performance data and subjective feedback were collected. Results show that touchless interaction outperformed touch interaction in depth-related tasks, while touch interaction was superior in lateral tasks. Subjective feedback indicated that touchless input scored higher in attractiveness, stimulation, and novelty. These findings provide guidance for selecting appropriate interaction modes in virtual museums and digital heritage.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Interactive and Immersive Displays · Hand Gesture Recognition Systems · Tactile and Sensory Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (56)
Show source-provided references
1. Caggianese, G., Gallo, L. & Neroni, P. Evaluation of spatial interaction techniques for virtual heritage applications: A case study of an interactive holographic projection. Future Gener. computer Syst. 81, 516–527 (2018).10.1016/j.future.2017.07.047 ↗
2. Tesoriero, R., Gallud, J. A., Lozano, M. & Penichet, V. M. R. Enhancing visitors’ experience in art museums using mobile technologies. Inf. Syst. Front. 16, 303–327 (2014).10.1007/s10796-012-9345-1 ↗
3. Gerba, C. P., Wuollet, A. L., Raisanen, P. & Lopez, G. U. Bacterial contamination of computer touch screens. Am. J. Infect. control 44, 358–360 (2016).10.1016/j.ajic.2015.10.013 ↗
4. Ionescu, D., Ionescu, B., Gadea, C. & Islam, S. Multimodal control of virtual game environments through gestures and physical controllers. In 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings, 1–6 (IEEE, 2011).10.1109/VECIMS.2011.6053846 ↗
5. Alexander, J., Barton, J. & Goeser, C. Transforming the art museum experience: Gallery one. In Museums and the Web (2013).
6. Baraldi, L., Paci, F., Serra, G., Benini, L. & Cucchiara, R. Gesture recognition using wearable vision sensors to enhance visitors’ museum experiences. IEEE Sens. J. 15, 2705–2714 (2015).
7. Tian, F. et al. An empirical study of virtual museum based on dual-mode mixed visualization: the sanxingdui bronzes. Herit. Sci. 12, 146 (2024).10.1186/s40494-024-01241-1 ↗
8. Bottino, A., Martina, A., Strada, F. & Toosi, A. Gaine–a portable framework for the development of edutainment applications based on multitouch and tangible interaction. Entertain. Comput. 16, 53–65 (2016).10.1016/j.entcom.2016.04.001 ↗
9. Bai, H., Gao, L., El-Sana, J. & Billinghurst, M. Markerless 3d gesture-based interaction for handheld augmented reality interfaces. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, 1–1 (2013).10.1145/2543651.2543678 ↗
10. Knoedel, S. & Hachet, M. Multi-touch rst in 2d and 3d spaces: Studying the impact of directness on user performance. In 2011 IEEE Symposium on 3D User Interfaces (3DUI), 75–78 (IEEE, 2011).10.1109/3DUI.2011.5759220 ↗
11. Bachmann, D., Weichert, F. & Rinkenauer, G. Evaluation of the leap motion controller as a new contact-free pointing device. Sensors 15, 214–233 (2014).10.3390/s150100214 ↗
12. Ivory, M. Y. & Hearst, M. A. The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. (CSUR) 33, 470–516 (2001).10.1145/503112.503114 ↗
13. Pallud, J. Impact of interactive technologies on stimulating learning experiences in a museum. Inf. Manag. 54, 465–478 (2017).10.1016/j.im.2016.10.004 ↗
14. vom Lehn, D., Hindmarsh, J., Luff, P. & Heath, C. Engaging constable: revealing art with new technology. In Proceedings of the SIGCHI conference on Human factors in computing systems, 1485–1494 (2007).10.1145/1240624.1240848 ↗
15. Kopper, R., Ni, T., Bowman, D. A. & Pinho, M. Design and evaluation of navigation techniques for multiscale virtual environments. In Ieee virtual reality conference (vr 2006), 175–182 (Ieee, 2006).
16. Barbieri, L., Bruno, F. & Muzzupappa, M. Virtual museum system evaluation through user studies. J. Cultural Herit. 26, 101–108 (2017).10.1016/j.culher.2017.02.005 ↗
17. Barneche-Naya, V. & Hernández-Ibáñez, L. A. Ux aspects of kinect-based movement schemes inside virtual environments for museum installations. In Learning and Collaboration Technologies. Ubiquitous and Virtual Environments for Learning and Collaboration: 6th International Conference, LCT 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Part II 21, 133–150 (Springer, 2019).10.1007/978-3-030-21817-1_11 ↗
18. Manghisi, V. M. et al. Enhancing user engagement through the user centric design of a mid-air gesture-based interface for the navigation of virtual-tours in cultural heritage expositions. J. Cultural Herit. 32, 186–197 (2018).10.1016/j.culher.2018.02.014 ↗
19. De Paolis, L. T. & De Luca, V. The effects of touchless interaction on usability and sense of presence in a virtual environment. Virtual Real. 26, 1551–1571 (2022).10.1007/s10055-022-00647-1 ↗
20. Kyriakou, P. & Hermon, S. Can i touch this? using natural interaction in a museum augmented reality system. Digital Appl. Archaeol. Cultural Herit. 12, e00088 (2019).
21. Carrozzino, M. & Bergamasco, M. Beyond virtual museums: Experiencing immersive virtual reality in real museums. J. Cultural Herit. 11, 452–458 (2010).10.1016/j.culher.2010.04.001 ↗
22. Peinado-Santana, S., Hernández-Lamas, P., Bernabéu-Larena, J., Cabau-Anchuelo, B. & Martín-Caro, J. A. Public works heritage 3d model digitisation, optimisation and dissemination with free and open-source software and platforms and low-cost tools. Sustainability 13, 13020 (2021).10.3390/su132313020 ↗
23. Ren, G., Li, C., O’Neill, E. & Willis, P. 3d freehand gestural navigation for interactive public displays. IEEE Computer Graph. Appl. 33, 47–55 (2013).10.1109/MCG.2013.15 ↗
24. Burigat, S. & Chittaro, L. Navigation in 3d virtual environments: Effects of user experience and location-pointing navigation aids. Int. J. Hum.-Computer Stud. 65, 945–958 (2007).10.1016/j.ijhcs.2007.07.003 ↗
25. Liu, J. Development of interactive english e-learning video entertainment teaching environment based on virtual reality and game teaching emotion analysis. Entertain. Comput. 52, 100884 (2025).10.1016/j.entcom.2024.100884 ↗
26. Song, M., Elias, T., Martinovic, I., Mueller-Wittig, W. & Chan, T. K. Digital heritage application as an edutainment tool. In Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry, 163–167 (2004).10.1145/1044588.1044621 ↗
27. Zaharias, P., Michael, D. & Chrysanthou, Y. Learning through multi-touch interfaces in museum exhibits: An empirical investigation. J. Educ. Technol. Soc. 16, 374–384 (2013).
28. Motion, L. Leap motion controller. https://www.leapmotion.com Accessed: 2024-10-21 (2015).
29. Weichert, F., Bachmann, D., Rudak, B. & Fisseler, D. Analysis of the accuracy and robustness of the leap motion controller. Sensors 13, 6380–6393 (2013).10.3390/s130506380 ↗
30. Yang, E. & Dorneich, M. C. The effect of time delay on emotion, arousal, and satisfaction in human-robot interaction. In Proceedings of the human factors and ergonomics society annual meeting, vol. 59, 443–447 (SAGE Publications Sage CA: Los Angeles, CA, 2015).10.1177/1541931215591094 ↗
31. apple. Use an ipad as a second display for a mac. https://support.apple.com/en-us/102597 Accessed: 2024-10-21 (2024).
32. ISO, I. Ergonomics of human-system interaction-part 11: Usability: Definitions and concepts (iso 9241-11: 2018). Google Scholar Google Scholar Reference (2018).
33. Watson, B., Walker, N., Ribarsky, W. & Spaulding, V. Effects of variation in system responsiveness on user performance in virtual environments. Hum. Factors 40, 403–414 (1998).10.1518/001872098779591287 ↗
34. Bowman, D. A., Koller, D. & Hodges, L. F. A methodology for the evaluation of travel techniques for immersive virtual environments. Virtual Real. 3, 120–131 (1998).10.1007/BF01417673 ↗
35. Griffiths, J. R., Johnson, F. & Hartley, R. J. User satisfaction as a measure of system performance. J. Librariansh. Inf. Sci. 39, 142–152 (2007).10.1177/0961000607080417 ↗
36. Lund, A. M. Measuring usability with the use questionnaire12. Usability interface 8, 3–6 (2001).
37. Brooke, J. et al. Sus-a quick and dirty usability scale. Usability Eval. Ind. 189, 4–7 (1996).
38. Laugwitz, B., Held, T. & Schrepp, M. Construction and evaluation of a user experience questionnaire. In HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, November 20-21, 2008. Proceedings 4, 63–76 (Springer, 2008).
39. Caggianese, G., Gallo, L. & De Pietro, G. Design and preliminary evaluation of a touchless interface for manipulating virtual heritage artefacts. In 2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems, 493–500 (IEEE, 2014).10.1109/SITIS.2014.44 ↗
40. Shao, Y. Study on the historical development of Ginghua art in Jingdezhen (1949-1966). Art. Des. Rev. 11, 346–362 (2023).
41. Pollefeys, M. et al. Acquisition of detailed models for virtual reality. Bar Int. Ser. 843, 71–78 (2000).
42. Xiaoqi, F. The plum vase with intertwined floral patterns. https://www.dpm.org.cn/collection/ceramic/226906. Accessed: 2024-10-28.
43. Wei, X. The vault-of-heaven vase with intertwined floral patterns. https://www.dpm.org.cn/collection/ceramic/226869. Accessed: 2024-10-28.
44. Yi, C. The dragon-pattern jar. https://www.dpm.org.cn/collection/ceramic/226863.html. Accessed: 2024-10-28.
45. Blender. Blender official website. https://www.blender.org/. Accessed: 2024-10-18.
46. Procreate. Procreate official website. https://procreate.com/procreate. Accessed: 2024-10-18.
47. Unity Technologies. Unity xr management documentation. https://docs.unity3d.com/6000.0/Documentation/Manual/com.unity.xr.management.html. Accessed: 2024-07-13.
48. Kohli, D. & Hsu, M.-S. The Jacobian analysis of workspaces of mechanical manipulators. Mechanism Mach. Theory 22, 265–275 (1987).10.1016/0094-114X(87)90010-3 ↗
49. Hachet, M., de La Rivière, J.-B., Laviole, J., Cohé, A. & Cursan, S. Touch-based interfaces for interacting with 3d content in public exhibitions. IEEE Computer Graph. Appl. 33, 80–85 (2013).10.1109/MCG.2013.34 ↗
50. Pukari, A., Nivalainen, J., Alavesa, P. & Korkiakoski, M. Learn flags: Assessment of the learning curve of inexperienced users with gesture interactions with hands-free augmented reality. In Proceedings of the 26th International Academic Mindtrek Conference, 340–343 (2023).10.1145/3616961.3617811 ↗
51. Hincapié-Ramos, J. D., Guo, X., Moghadasian, P. & Irani, P. Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1063–1072 (2014).10.1145/2556288.2557130 ↗
52. Hansberger, J. T. et al. Dispelling the gorilla arm syndrome: the viability of prolonged gesture interactions. In Virtual, Augmented and Mixed Reality: 9th International Conference, VAMR 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings 9, 505–520 (Springer, 2017).10.1007/978-3-319-57987-0_41 ↗
53. Kawabe, T. & Ujitoko, Y. Softness perception of visual objects controlled by touchless inputs: The role of effective distance of hand movements. IEEE Trans. Vis. Computer Graph. 30, 4154–4169 (2024).10.1109/TVCG.2023.3254522 ↗
54. Kawabe, T., Ujitoko, Y. & Yokosaka, T. The relationship between illusory heaviness sensation and the motion speed of visual feedback in gesture-based touchless inputs. Front. Psychol. 13, 811881 (2022).10.3389/fpsyg.2022.811881 ↗
55. Lee, J. D. L., Restoles, D. M. D. & Vidal Jr, E. C. E. A virtual reality object interaction system with complex hand interactions. In International Conference on Human-Computer Interaction, 122–134 (Springer, 2023).10.1007/978-3-031-48050-8_9 ↗
56. Loup-Escande, E. et al. Contributions of mixed reality in a calligraphy learning task: Effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance. Computers Hum. Behav. 75, 42–49 (2017).10.1016/j.chb.2017.05.006 ↗