Monitoring Driver’s Visual and Mental Workload

Introduction

Ever since radios entered to the cars, the drivers have been obliged to share their attention to other tasks than driving. Nowadays the number of devices inside the cockpit is evermore increasing with Advanced Driver Assistance Systems (ADASs), In Vehicle Information Systems (IVISs) and nomadic devices, e.g. personal digital assistants (PDAs) and mobile phones. However, the driver may not always be in the situation to pay attention to the messages from the various systems. Hence, it is important and of interest to maintain the driver’s distraction level within reasonable limits. This can be done with technical solutions which by monitoring the driver and his behaviour are able to manage and prioritize the information from the different devices. The EU-funded project Adaptive Integrated Driver-vehicle interface (AIDE) [1] was released to solve such problems by developing the necessary technologies.

The AIDE architecture consists of five independent Driver-Vehicle- Environment (DVE) modules. They are the Traffic and Environment Risk Assessment (TERA), the Driver Characteristics (DC), the Driver State Degradation (DSD), the Cockpit Activity Assessment (CAA) and the Driver Availability Estimator (DAE) module. The modules use information from a stereo-vision system, CAN bus and a navigation system for their specific tasks.

VTT developed the CAA module which focused on the driverís level of distraction caused by secondary tasks (e.g. phone calls, operating radio, etc.) together with Volvo Technology. The prototype implementation utilises faceLAB of Seeing Machines [2] for tracking the driver-related parameters, such as head and gaze positions and orientations. The system will be evaluated in the Volvoís truck and the SEAT’s passenger car during Spring 2007.

Visual and cognitive distraction

Two types of distraction were monitored: visual and cognitive. Visual distraction (VD) means the situations when the driver is not looking at the road but in-vehicle and outside targets such as traffic signs and dashboard (’’eyes-off-road’’) or the driver is sharing his/her attention between road and another target (e.g. mirror) by making continuous short glances. The cognitive distraction (CD) covers the cases when driver’s mind is not entirely concentrated on the driving task (’’mind-off-road’’). Such situations occur when the driver is making conversation, daydreaming or hard thinking.

For detecting the visual distraction, the view inside a cockpit was divided into four clusters: windscreen, road ahead, left and right exterior mirror. They are presented in Figure 1. The windscreen cluster covers the whole view through the windshield glass. As the road is mostly seen through the windshield, the road ahead cluster is located inside the borders of the windscreen cluster. Logically, the exterior mirror clusters are located at both sides of the windscreen.

The decision boundaries of clusters were created manually by observing driver’s behaviour and gaze direction from pre-recorded videos. It is not possible to use same set of the clusters for both head and eye movements, therefore two separate sets were generated. Pri- marily, the eye movements are used for detection but in that case when eye-tracking is lost the decision is based on the head movements.

Results

The attention mapping clusters follow and detect the targets of the driver’s gaze rather reliably. Occasionally glances are missed or falsely detected but considering the taskís difficulty with fast situation changes and multiple different drivers the cluster set worked at least moderately. The average performances of the clusters were 76% for the road ahead, 56% for the left mirror and 42% for the right mirror. No severe differences in classification performances were noticed between different driving environments whereas performance varied along with the driver as expected.

Monitoring the driver’s cognitive workload resulted to fine classification performance which – as the visual distraction detection – depends on the driver. The performances between two vehicles did not differ significantly even though lane position sensor was not available in the passenger car.

In overall, the cognitive distraction detection works nicely and it has provided promising results. Supposedly the drivers were not very highly cognitively loaded during the test data gathering because the cognitive tasks were not very demanding or the drivers may have concentrated more on the driving task than normally. Nevertheless, even under these conditions the cognitive distraction of the drivers was well and reliably detected. The classifier achieved approximately 68% classification performance in the truck tests and even 86 % for the city car.

In the near-future, the models are evaluated within real driving environment to learn how well they adapt to different drivers and situations. Meanwhile, we will accentuate our experience for working with traffic safety-related applications utilizing machine vision systems especially in the fields of driver and environment monitoring.

References
[1] AIDE web page, www.aide-eu.org (referred in 15 Feb 2007)
[2] Seeing Machines web page, www.seeingmachines.com (referred in 15 Feb 2007)
[3] SVMlight web page, svmlight.joachims.org (referred in 15 Feb 2007)

Contact Information: Digital Information Systems Ms. Maria Jokela Tel. +358 20 722 3691 Dr, Matti Kutila Tel. +358 20†722 3619 matti.kutila@vtt.fi www.vtt.fi