Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise

Jeesun Kim, Sonya Karisma, Vincent Aubanel and Chris Davis

Abstract:

The speech of a familiar talker is better recognized in noise than an unfamiliar one, suggesting that listeners access talker-specific models to assist with degraded input. This study investigated whether a talker model could be accessed by presenting the face of a talker. In the experiment, participants were trained in recognizing three talkers’ faces and voices to ceiling-level. Participants were then given a speech in noise recognition task consisting of four talker conditions: familiar face then familiar voice; unfamiliar face then familiar voice, familiar face then unfamiliar voice; and unfamiliar face then unfamiliar voice. A talker familiarity effect was found, i.e., speech perception was more accurate in the familiar face and familiar voice condition than all other ones. A familiar voice did not produce a talker familiarity effect when paired with an unfamiliar face. The familiar face and unfamiliar voice condition had the poorest performance, indicating that pairing a familiar face and unfamiliar voice had a disruptive effect. The results suggest that listeners develop a talker model that includes details of both the voice and the face; and that accessing this model can in some circumstances be wholly determined by face cues.

---

Cite as: Kim, J., Karisma, S., Aubanel, V., Davis, C. (2018) Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise. Proc. Interspeech 2018, 2276-2279, DOI: 10.21437/Interspeech.2018-1812.

---

BiBTeX Entry:

@inproceedings{Kim2018,
author={Jeesun Kim and Sonya Karisma and Vincent Aubanel and Chris Davis},
title={Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise},
year=2018,
booktitle={Proc. Interspeech 2018},
pages={2276--2279},
doi={10.21437/Interspeech.2018-1812},
url={http://dx.doi.org/10.21437/Interspeech.2018-1812} }