Speechreading Supplemented by Single-Channel and Multichannel Tactile Displays of Voice Fundamental Frequency

The benefits of two tactile codes of voice fundamental frequency (F₀) were evaluated as supplements to the speechreading of sentences in two short- term training studies, each using 12 adults with normal hearing. In Experiment 1, a multichannel spatiotemporal display of F₀, known as Portapitch, was used to stimulate the index finger. In an attempt to improve on past performance with this display, the coding scheme was modified to better cover the F₀ range of the talker in the training materials. For Experiment 2, to engage kinesthetic/proprioceptive pathways, a novel single-channel positional display was built, in which F₀ was coded as the vertical displacement of a small finger-rest. Input to both displays consisted of synthesized replicas of the F₀ contours of the sentences, prepared and perfected off-line. Training with the two tactile F₀ displays included auditory presentation of the synthesized F₀ contours in conjunction with the tactile patterns on alternate trials. Speechreading enhancement by the two tactile F₀ displays was compared to the enhancement provided when auditory F information was available in conjunction with the tactile patterns, by auditory presentation of a sinusoidal indication of the presence or absence of voicing, and by a single-channel tactile display of the speech waveform presented to the index finger. Despite the modified coding strategy, the multichannel Portapitch provided a mean tactile speechreading enhancement of 7 percentage points, which was no greater than that found in previous studies. The novel positional F₀ display provided only a 4 percentage point enhancement. Neither F₀ display was better than the simple single-channel tactile transform of the full speech waveform, which gave a 7 percentage point enhancement effect. Auditory speechreading enhancement effects were 17 percentage points with the voicing indicator and approximately 35 percentage points when the auditory F₀ contour was provided in conjunction with the tactile displays. The findings are consistent with the hypothesis that subjects were not taking full advantage of the F₀ variation information available in the outputs of the two experimental tactile displays.

KEY WORDS: speechreading, lipreading, tactile aids, speech perception, fundamental frequency

Submitted on December 6, 1993
Accepted on January 9, 1995

CiteULike    Connotea    Delicious    Digg    Facebook    Reddit    Technorati    Twitter    What's this?