Multimodal human-computer interaction for virtual reality

Research Article
Open access

Multimodal human-computer interaction for virtual reality

Jiajun Tang 1* , Manni Gong 2 , Shangxi Jiang 3 , Yiji Dong 4 , Taojie Gao 5
  • 1 Shool of Advanced Technology Xi'an Jiaotong-liverpoor Unsiversity Suzhou, China    
  • 2 Wuhan Britain-China School    
  • 3 Suzhou Science &Technology Town Foreign Language School    
  • 4 Jinan Foreign Language School    
  • 5 Wuhan Britain- China School     
  • *corresponding author Jiajun.tang21@student.xjtlu.edu.cn
Published on 23 February 2024 | https://doi.org/10.54254/2755-2721/42/20230778
ACE Vol.42
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-309-8
ISBN (Online): 978-1-83558-310-4

Abstract

Virtual reality (VR) is an interactive simulation environment based on computer technology that enables users to immerse themselves in a virtual world. With the continuous advancement of technology, VR has shown great potential in various fields. This article aims to review the application of VR in different fields and its impact on society. First, this paper introduces VR's basic principles and technical characteristics. VR systems are usually composed of head-mounted display devices, trackers, joysticks, etc., enabling users to experience immersive virtual environments by simulating real-world perception and interaction. Secondly, this paper discusses the application of VR in the field of education. VR technology can provide an immersive learning experience, enabling students to personally participate in the course content, greatly enhancing the effect of learning. At the same time, VR can also simulate experimental scenes, train operational skills, etc., to provide students with more abundant and practical learning resources. Then, this paper introduces the application of VR in the medical field. VR can be used for rehabilitation training of patients, psychological treatment, and other aspects, effectively improving the quality and effect of medical services. In addition, this paper also discusses the application of VR in entertainment, tourism, and other fields. VR technology can bring users immersive gaming experience, virtual tourism, etc., and inject new vitality into the entertainment industry. In tourism, VR can provide tourists with an immersive sightseeing experience, so they can feel the distant scenery at home.

Keywords:

multimodal interaction, virtual reality, styling, human computer interaction, eye tracking

Tang,J.;Gong,M.;Jiang,S.;Dong,Y.;Gao,T. (2024). Multimodal human-computer interaction for virtual reality. Applied and Computational Engineering,42,201-207.
Export citation

References

[1]. B. Federick.” Myron Krueger.” PHWORKS.com.http://thedigitalage.pbworks.com/w/page/22039083/Myron%20Krueger (accessed Nov. 23, 2009).

[2]. E. Bozkir, “Towards Everyday Virtual Reality through Eye Tracking,” arXiv.org, Mar. 2022.

[3]. Han et al,” Patent analysis of augmented reality/virtual reality on head mounted display,” Hebei Journal of Industrial Science & Technology, vol. 34, no. 5, pp. 345–353, 2017.

[4]. P. Majaranta and A. Bulling, Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies. IGI Global, 2011.

[5]. A. D. Souchet, S. Philippe, D. Lourdeaux, and L. Leroy, “Measuring Visual Fatigue and Cognitive Load via Eye Tracking while Learning with Virtual Reality Head-Mounted Displays: A Review,” International Journal of Human–Computer Interaction, vol. 38, no. 9, pp. 1–24, Sep. 2021.

[6]. S. Stadler, H. Cornet, and F. Frenkler, “Assessing Heuristic Evaluation in Immersive Virtual Reality—A Case Study on Future Guidance Systems,” Multimodal Technologies and Interaction, vol. 7, no. 2, pp. 19, Feb. 2023.

[7]. X. Lou, X. A. Li, P. Hansen, and P. Du, “Hand-adaptive user interface: improved gestural interaction in virtual reality,” Virtual Reality, vol. 25, no. 2, pp. 367–382, Jul. 2020.

[8]. M. Lui, Kwok Ying Chong, M. Mullally, and R. McEwen, “Facilitated model-based reasoning in immersive virtual reality: Meaning-making and embodied interactions with dynamic processes,” International Journal of Computer-supported Collaborative Learning, vol. 18, no. 2, pp.203-230, Aug. 2023.

[9]. L.Cao, H. Zhang, C. Peng, and J. T. Hansberger, “Real-time multimodal interaction in virtual reality - a case study with a large virtual interface,” Multimedia Tools and Applications, vol. 82, no. 16, pp. 25427–25448, Feb. 2023.

[10]. R. Rao, T. Conn, S. Jung, J. Katupitiya, T. Kientz, V. Kumar, J. Ostrowski, S. Patel, C. Taylor, “Human robot interaction: application to smart wheelchairs,” Scholarly Commons (University of Pennsylvania), Jun. 2003.

[11]. D. Guttentag, “Virtual reality: Applications and implications for tourism,” Tourism Management, vol. 31, no. 5, pp. 637–651, Oct. 2010.

Cite this article

Tang,J.;Gong,M.;Jiang,S.;Dong,Y.;Gao,T. (2024). Multimodal human-computer interaction for virtual reality. Applied and Computational Engineering,42,201-207.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation

ISBN：978-1-83558-309-8(Print) / 978-1-83558-310-4(Online)
Editor：Mustafa İSTANBULLU
Conference website: https://2023.confmla.org/
Conference date: 18 October 2023
Series: Applied and Computational Engineering
Volume number: Vol.42
ISSN：2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. B. Federick.” Myron Krueger.” PHWORKS.com.http://thedigitalage.pbworks.com/w/page/22039083/Myron%20Krueger (accessed Nov. 23, 2009).

[2]. E. Bozkir, “Towards Everyday Virtual Reality through Eye Tracking,” arXiv.org, Mar. 2022.

[3]. Han et al,” Patent analysis of augmented reality/virtual reality on head mounted display,” Hebei Journal of Industrial Science & Technology, vol. 34, no. 5, pp. 345–353, 2017.

[4]. P. Majaranta and A. Bulling, Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies. IGI Global, 2011.

[5]. A. D. Souchet, S. Philippe, D. Lourdeaux, and L. Leroy, “Measuring Visual Fatigue and Cognitive Load via Eye Tracking while Learning with Virtual Reality Head-Mounted Displays: A Review,” International Journal of Human–Computer Interaction, vol. 38, no. 9, pp. 1–24, Sep. 2021.

[6]. S. Stadler, H. Cornet, and F. Frenkler, “Assessing Heuristic Evaluation in Immersive Virtual Reality—A Case Study on Future Guidance Systems,” Multimodal Technologies and Interaction, vol. 7, no. 2, pp. 19, Feb. 2023.

[7]. X. Lou, X. A. Li, P. Hansen, and P. Du, “Hand-adaptive user interface: improved gestural interaction in virtual reality,” Virtual Reality, vol. 25, no. 2, pp. 367–382, Jul. 2020.

[8]. M. Lui, Kwok Ying Chong, M. Mullally, and R. McEwen, “Facilitated model-based reasoning in immersive virtual reality: Meaning-making and embodied interactions with dynamic processes,” International Journal of Computer-supported Collaborative Learning, vol. 18, no. 2, pp.203-230, Aug. 2023.

[9]. L.Cao, H. Zhang, C. Peng, and J. T. Hansberger, “Real-time multimodal interaction in virtual reality - a case study with a large virtual interface,” Multimedia Tools and Applications, vol. 82, no. 16, pp. 25427–25448, Feb. 2023.

[10]. R. Rao, T. Conn, S. Jung, J. Katupitiya, T. Kientz, V. Kumar, J. Ostrowski, S. Patel, C. Taylor, “Human robot interaction: application to smart wheelchairs,” Scholarly Commons (University of Pennsylvania), Jun. 2003.

[11]. D. Guttentag, “Virtual reality: Applications and implications for tourism,” Tourism Management, vol. 31, no. 5, pp. 637–651, Oct. 2010.

For authors
Guide for authors Policies for authors Aims and scope Subscription Open access policy
For editors
Policies for editors Editorial board
For reviewers
Policies for reviewers Privacy policy

Cookies are used by this site.
Copyright © 2024 EWA Publishing, its licensors, and contributors. All rights reserved, including text and data mining, AI training, and similar technologies.