1. Introduction
According to statistics, 1.3 million preventable deaths die in traffic accidents every year [1]. Traffic accidents are also the most important cause of death among young people. The WHO claims that 13 million vehicles will cause the 13 million deaths over the next ten years [1]. Many exogenous human behaviors lead to road injury and death. Common factors include speeding, drunk driving, fatigued driving, distracted driving, and not wearing a seat belt [1-4]. Speeding, drunk driving, wearing seatbelts, and distracted driving are all spontaneous and illegal in some regions and countries. Distracted driving arises due to social media, eating, navigation and talking with passengers [5]. Moreover, this work shows that high-speed areas are not the leading cause of fatigue and distraction. In on-board technology [6], with the development of automatic driving and gradually perfect, more and more drivers choose to buy autonomous car travel, due to the automated driving auxiliary driving function has been by the first level and secondary upgrade to now level 3 extra autopilot, according to the association of automotive engineers of auto automation ability ranking and definition, level 3 is defined as the condition of automatic driving. In other words, level 3 vehicles take over the car only in particular circumstances and remain autonomous for the rest of the time. Taking over operation is the driver’s interaction with the car [7]. The process is driven by desire, environmental network limitations , and system capacity limitations. The process is divided into three main phases, transition, control , and avoidance[8]. So it can be simply explained that when the autopilot system detects that the car is about to hit an obstacle and the driver must immediately control the vehicle [9], it sends a take-over request (TOR) to the driver, and the driver then controls the car. Many researchers are optimistic about level 3 autonomous cars and think they are safer than manual vehicles. However, according to several studies, the security operation of autonomous vehicles still depends on drivers taking over operations and trust. So the problem follows, when the car takes over the process of driving, most drivers will not stay highly concentrated, will produce fatigue and distraction, so in this case, if encounter an emergency, the driver is not timely received take over the request and take over the vehicle, will produce dangerous, so someone based on this situation based on level 3 automatic driving for driver fatigue distraction warning system [10, 11]. This article will review the design of the above system.
2. Main body
2.1. System design.
2.1.1. Human-centered design process. In most researchers' system designs, the Human-Centered Design (HCD) method is used. To make the designed HMI system more in line with the requirements and expectations of drivers and stakeholders, the HCD method will focus on the technical Design of the final user, consider driving task organization and environment; HCD process requires end users and stakeholders to participate in the whole system design process [12]. As shown in Figure 1, in the people-centered design process, Three things will happen first before designing, They are, respectively, literature review, context of use description, and Initial truck driver interviews; the analysis and review of the last literature, Find out the similarities and differences with previous system designs, based on the original study, to improve your system design views and learn what other designers think, Then give access to the driver, The access to the driver's ideas and preferences as the end user, And can draw the professional opinion as a driver. Then, according to the above three operations, analysis and get a user requirements table, and then the designer can according to the result of this table, according to the user demand, improve the system design, and after the system design, the selected driver road test, and after testing them, collect their feedback and opinions. Figure 1 Display is a complete people-oriented design process. In this process, the end users, stakeholders and their ideas are involved in the whole process, and the system is designed according to their needs, while achieving people-oriented, and to a certain extent, it can ensure the sales volume of the system in commercial use.
Figure 1. HCD method for in-vehicle drivers warning system development [11].
A vital advantage of the HCD process is that it is unvarying; that is, it is not a linelike process, remaining unchanged from concept to production[13]. So this process will involve the access and access to the end users and stakeholders, and according to the access or survey results improve and adjustment of the design, and then based on the adjustment and improved system again access and investigation to the end users and stakeholders, get the final feedback, is the end users and stakeholders involved, embodies the core ideas of the text, in this process, through the interaction and communication with the user, also can make the designed system more in line with the expectations of the end users and stakeholders.
During interviews with end users and stakeholders before system design,the following ten requirements were identified:
1)The device should use a ‘graduated’ approach (warnings hierarchy) for fatigue and distraction warnings.
2)The device should display the information effectively to drivers.
3)The device should have continuous displays of fatigue/distraction?
4)The device should require driver input?
5)The device should timely display information to drivers.
6)The device should be tamper-proof and not be modifiable.
7)The device should be located in a position that does not cause additional distraction or visual demand.
8)The device should not interfere with other in-cab systems.
9)The device should not require extensive training to understand and use the system.
10)The device should be trusted and accepted by drivers.
And then , researchers could design the HMI system according to the table.
2.1.2. Warning system design
In the design of the HMI system analyzed in this paper, most systems apply multimodal warning systems to warn tired or distracted drivers of the three aspects of hearing, vision , and touch. And the system is an increasing multimodal warning according to the fatigue or distraction of the driver of the autonomous driving process detected by the detection system. And give different warnings about fatigue and distraction by seeing the results of fatigue or distraction, at the degree of driver fatigue or distraction is mild, triple alarm, HUD and seat periodic pulses. When the degree of driver fatigue or distraction is moderate. Warning with louder alarms, HUD, and seat intermittent pulses, when the driver's fatigue or distraction process is severe, max volume alarm, HUD and whole seat vibration.After these three degrees of warning processes. If the detection system detects that the driver's state is consciousness after a paticular stage of warning and takes over the driving from the level 3 autonomous process. The warning system will be lifted. If detected that the driver is not yet awake or has not taken over the car. Will move from the warning of that stage to the next level of caution, until the driver is sober and takes over the car's manual driving [11].
2.1.3. Design of detection system
In the system conceived of the study analyzed in this paper, the detection system is all designed with the eye-tracking system. Data from the pupil of the eye tracker, and recorded with an eyeshadow camera mounted monitor in front of the subject. The system memorializes the data [10] of the pupil region every second, the one-second interval is designed to ensure that when analyzing the data from the pupil region of the driving eye, that the physiological state of the driver in a short time will not deviate significantly from the date of the last analysis, and if the following data acquired changes, it can be immediately analyzed and understood, it detects the pupil area of the driver almost in real-time to detect whether the physiological state of the autonomous car driver is fatigue or distraction. Meanwhile, a camera will also be placed near the driver's feet to see the movement of the feet of the independent car driver, to cooperate with the eye shadow camera to jointly see the fatigue or distraction state of the independent car driver [10]. Foot near the camera, will determine whether the driver’s foot action changes, to determine whether the driver state is fatigued or distracted, but just judge whether there is an action and not accurately judge the state of the driver, may driver in a sleep state will have foot movement, may the driver in awake foot movement without change, so still need to combine the data analysis and pupil area.
2.2. System evaluation
After the completion of the system design, different research literature used two different methods for the evaluation of the HMI system design from two perspectives. User-based tests and check-based tests, respectively. In one of the literature, the investigators selected the creation of the system, and asked them to test in the cab of the system. The test is summarized as follows. Based on the users’ assessment; every participant has tested separately: 14 drivers, including 11 males and three females; The drivers average age is 38.4 years old , and the average driving experience of about 18 years . All participants reached the simulator facility, offered informed consent, and finished a demographic and driving experience examination. They got an outline of the study. Participants were in the company to the car simulator and construction in the cab. For the simulator experiments, a 5-min familiarization drive was first given. The trial began when the participants were satisfied with the simulator and alerted the driver at a predefined break. The assessment does not measure the driver's behavior, but focuses on the warning for understanding, acceptance, perceived validity, and the driver's opinion. The HMI was accepted and considered useful by the participant group, which is based on users, including experienced drivers. Diverse warnings in the MOH were noted, correctly understood, and thought valid by the participants. Likewise, the participants were considered unlikely to interfere with signs in other taxis excessively and to surcharge or scare the driver. HMIs in reviews based on inspection also pass two checklist / usability principles. The participants did not find any significant design flaws. The recommended design iterations for the two assessments are: including a system status button. The note allows adjustment of auditory or visual warnings; circadian patterns including a visible sign. Improve the size of the acoustic tone and sound because it is not frequently sensed by the noise in the background [11].
After the test, through the form of the questionnaire, the view of the system, and the data summary, get the final result, HMI subjects considered useful [10]. Others are inspection-based tests, in which false alarms may occur, but no other significant defects were found [11].
3. Discussion
Under the analysis of the above system design, most of the literature adopts the HCD (people-oriented) design thought core, so as to be more in line with the user's expectations and ideas, the specific implementation of the design system first access to the end users and stakeholders, and the selection after the system selection car driver vehicle test, and through the questionnaire in the form of the test after the driver investigation, and concluded. In terms of system design, most of the literature is taken by detecting the data of the pupil area and the changes of foot movement as a detection system to determine whether the driver fatigue or distraction and the degree of fatigue or distraction, and sent to the driver takeover request (TOR).Then after the detection of data analysis into results, if the result is distraction or fatigue, then the alarm system will be activated. Alert approach to auditory, touch, visual , multimodal design, but also according to the driver's state and restore awake degree strength gradually increasing multimodal warning, and from two aspects of fatigue and distraction, two different kinds of notice, implements the comprehensive caution, increase the wake driver to promote the efficiency of the takeover process. In terms of system evaluation, is divided into two kinds; one is based on a user test, which conforms to the core idea of people-oriented (HCD); after the test, the user questionnaire form of survey and inquiry, helps to find their vulnerabilities and upgrade and improve the system, one of the literatures in the study of respondent’s questionnaire form of the results for the HMI system. The other is a check-based test, in which false alarms occur during the test, and the rest perform well. Therefore, according to the above, in the literature, checking the driver status still has a significant error leading to the occurrence of false alarm, and the alarm level is level 3.However, the system design process did not consider if after the most vital level alert, the driver if still not recover awake and take over the car, what measures will be taken to ensure further that the driver to recover awake and promote the takeover process.
4. Conclusion
This review reviews the existing alarm systems designed for driver fatigue and distraction during level 3 autonomous driving. The alarm system for most of the literature adopts the core idea of people-oriented (HCD) design. That is, the review of previous literature. Get experience from this, the context of use description And interviews with drivers before the design. Then based on these three operations. And design the system from this point, This is all about to better involve end users and stakeholders in the design process, Better in line with the core idea of people-oriented. And adopt multimodal alarm to awaken the driver, from hearing, visual, touch, and progressive, according to the automatic driver physiological state by fatigue or distraction to awake or keep fatigued or distracted, to stop the operation of warning or gradually advanced warning strength, the system will detect fatigue and distraction state as two different states, divided into two ways, starting from two points according to different situations, do for automatic driver state design warning system. In the literature reviewed in this paper. According to the core idea of putting people first. Road tests on some of the selected drivers after the design of the system. And collected feedback from the driver after the trail, the result display. System was considered helpful by the subjects, Be able to partly awaken drivers under fatigue or distraction and promote the take-over process to prevent accidents, however, the system design in some literature also has loopholes in detection error and alarm subsequent insurance measures. No researchers in the literature reviewed in this paper have addressed such issues, Therefore, in-depth research and improvement are needed.
References
[1]. Decade of Action for Road Safety 2021-2030.World Health Orgnazationn. 2021.
[2]. Liu, Z. Sun, N, Yang, T. Fang, Y, Optimal collaborative motion planning of dual boom cranes for transporting payloads to desired positions and attitudes. IEEE Transactions on Intelligent Transportation Systems. 2023, 24(6), 6096-6110.
[3]. Du, H. Teng, S. Chen, H. et al. Chat with ChatGPT on intelligent vehicles, an IEEE TIV perspective. IEEE Transactions on Intelligent Vehicles.2023, 8(3), 2020-2026.
[4]. Gao, Y. Tong, W. Chen, W. et al. Chat with ChatGPT on interactive engines for intelligent driving. IEEE Transactions on Intelligent Vehicles. 2023, 8(3), 2034-2036.
[5]. Ansari, S., Naghdy, F., & Du, H, Human-machine shared driving: challenges and future directions. IEEE Transactions on Intelligent Vehicles. 2022, 7(3), 1.
[6]. Filtness, A.J., Armstrong, K.A., Watson, A., & Smith, S, Sleep-related vehicle crashes on low speed roads. Accident Analysis and Prevention. 2016, 99, 279–286.
[7]. Li, S., Blythe, P., Guo, W. & Namdeo, A. Investigating the effects of age and disengagement in driving on driver’s takeover control performance in highly automated vehicles. Transp. Plan. Technol .2019, 42, 470–497.
[8]. Guo, H., Zhang, Y., Cai, S., & Chen, X. Effects of level 3 automated vehicle drivers’ fatigue on their take-over behaviour: A literature review. J. Advanced Transportation, 2021, 1–12.
[9]. Du, N. Zhou, F. Pulver, E.M. et al., Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving, Transportation Research Part C: Emerging Technologies. 2020, 112, 78–87.
[10]. Kasuga, N. Tanaka, A. Miyaoka, K. Ishikawa, T. Design of an HMI system promoting smooth and safe transition to manual from Level 3 automated driving, Int. J. Intelligent Transportation Systems Research .2020,18:1–12.
[11]. Horberry, T. Mulvihill, C. Fitzharris, M. et al. Human-centered design for an In-vehicle truck driver fatigue and distraction warning system. IEEE Transactions on Intelligent Transportation Systems.2022, 23, 6.
[12]. Horberry, Burgess-Limerick, T. R. and Steiner, L. Human-centered design for mining equipment and new technology .2018.
[13]. Ergonomics of human-system interaction-part 210: Human-centred design for interactive systems .2019, 9241.
Cite this article
Sun,H. (2024). Fatigue and distraction warning system for autonomous vehicle drivers in the process of three-level autonomous driving. Applied and Computational Engineering,31,72-77.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).
References
[1]. Decade of Action for Road Safety 2021-2030.World Health Orgnazationn. 2021.
[2]. Liu, Z. Sun, N, Yang, T. Fang, Y, Optimal collaborative motion planning of dual boom cranes for transporting payloads to desired positions and attitudes. IEEE Transactions on Intelligent Transportation Systems. 2023, 24(6), 6096-6110.
[3]. Du, H. Teng, S. Chen, H. et al. Chat with ChatGPT on intelligent vehicles, an IEEE TIV perspective. IEEE Transactions on Intelligent Vehicles.2023, 8(3), 2020-2026.
[4]. Gao, Y. Tong, W. Chen, W. et al. Chat with ChatGPT on interactive engines for intelligent driving. IEEE Transactions on Intelligent Vehicles. 2023, 8(3), 2034-2036.
[5]. Ansari, S., Naghdy, F., & Du, H, Human-machine shared driving: challenges and future directions. IEEE Transactions on Intelligent Vehicles. 2022, 7(3), 1.
[6]. Filtness, A.J., Armstrong, K.A., Watson, A., & Smith, S, Sleep-related vehicle crashes on low speed roads. Accident Analysis and Prevention. 2016, 99, 279–286.
[7]. Li, S., Blythe, P., Guo, W. & Namdeo, A. Investigating the effects of age and disengagement in driving on driver’s takeover control performance in highly automated vehicles. Transp. Plan. Technol .2019, 42, 470–497.
[8]. Guo, H., Zhang, Y., Cai, S., & Chen, X. Effects of level 3 automated vehicle drivers’ fatigue on their take-over behaviour: A literature review. J. Advanced Transportation, 2021, 1–12.
[9]. Du, N. Zhou, F. Pulver, E.M. et al., Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving, Transportation Research Part C: Emerging Technologies. 2020, 112, 78–87.
[10]. Kasuga, N. Tanaka, A. Miyaoka, K. Ishikawa, T. Design of an HMI system promoting smooth and safe transition to manual from Level 3 automated driving, Int. J. Intelligent Transportation Systems Research .2020,18:1–12.
[11]. Horberry, T. Mulvihill, C. Fitzharris, M. et al. Human-centered design for an In-vehicle truck driver fatigue and distraction warning system. IEEE Transactions on Intelligent Transportation Systems.2022, 23, 6.
[12]. Horberry, Burgess-Limerick, T. R. and Steiner, L. Human-centered design for mining equipment and new technology .2018.
[13]. Ergonomics of human-system interaction-part 210: Human-centred design for interactive systems .2019, 9241.