Fuzzy Analysis of Speech Metrics to Estimate Crew Alertness

Abstract

A novel approach for estimating alertness levels from speech and tagging them with a reliability component has been developed. The Fatigue Quotient and Believability are both derived from the time series analysis of the speech signal in the communication between the operator and dispatch.

Operator attention is the most important human factor element for safe transportation operations. In addition to substance abuse, illness and intoxication fatigue is a major contributing factor to the decrease of attention. The goal of this study was to develop a means to detect and estimate fatigue levels of railroad operating personnel during on-duty hours. This goal continues to gain importance with new efforts from the government to expand rail transportation operations as a tool for high speed mass transportation in urban areas. Previous research has shown that sleeping disorders, reduced hours of rest and disrupted circadian rhythms lead to significantly increased fatigue levels which manifest themselves in alterations of speech patterns as compared to alert states of mind. In this study vocal indicators of fatigue are extracted from the speech signal and Fuzzy Logic is used to generate an estimate of the cognitive state of the train conductor. The output is tagged with a believability metric based on its behavior with respect to previous outputs and a fully alert state. Communication between the conductor and dispatch over radio provides an unobtrusive way of accessing the speech signal through existing speech infrastructure. The speech signal is discretized and processed through a digital signal processing algorithm, which extracts speech metrics from the signal that were determined to be indicative of fatigue levels. Speech metrics include, but are not limited to, speech duration, silence duration, word production rate, phrase gap duration, number of words per phrase and speech intensity. A fuzzy logic minimum inference engine maps the inputs to an output through an empirically determined rule base. The rule base and the associated membership functions were derived from batch mode and real time testing and the subsequent tuning of parameters to refine the detection of changes in patterns. To increase the validity and transparency of the output time series analysis is used to create the believability metric. A moving average filter eliminates the short term fluctuations and determines the long term trend of the output. A moving standard deviation estimation quantifies instantaneous fluctuations and provides a measure of the difference to a nominal alertness state. A real time version of the algorithm was developed and prototyped on a generic, low cost and scalable hardware platform. Rapid Prototyping was realized through the Matlab/Simulink xPC Target toolbox which allowed for instant real time code generation, testing and modification.

This testing environment together with batch mode testing was used to extensively test and fine tune parameters to improve the performance of the algorithm. A testing procedure was developed and standardized to collect data and tune the parameters of the algorithm. As a high level goal it was proven that the concept of digital signal processing and Fuzzy Logic can be utilized to detect changes in speech and estimate alertness levels from it. Furthermore, this study has proven that the framework to run such an analysis continuously as a monitoring function in locomotive cabins is feasible and can be realized with relatively inexpensive hardware. The development, implementation and testing process conducted for this project is explained and results are presented here.