Communication of emotions in everyday speech relies on integrating different types of information, including auditory and visual cues, such as words and facial expressions. While previous studies have examined the role of emotional words, less attention has been given to the effects of cross-modal interactions, such as the interplay between semantic content, speech prosody, and facial expressions. In our studies, we used various experiments to investigate the importance of prosody in emotion perception and how it affects both unimodal and multimodal emotion-matching tasks. In addition, our results suggest subtle but significant gender differences in emotion processing, which could affect understanding emotion and language processing in special populations, such as those with mental disorders like autism. Overall, these studies provide a foundation for future research in brain sciences and have potential applications in clinical settings.
Speaker
Professor Hongwei DING (Shanghai Jiaotong University)
Professor Hongwei Ding is a professor at the School of Foreign Languages, Shanghai Jiaotong University, with a background in linguistics (Dr. phil.) and speech technology (Dr.-Ing.habil.). She is the Principal Investigator of the Major Program of the National Social Science Foundation of China, and she serves as an officer of SProSIG, Vice Chairman of the Phonetics Branch of the Linguistic Society of China. Her current research interests focus on second language speech acquisition and speech prosody from a cognitive neuroscience perspective for individuals with hearing and mental impairments. Additionally, she has continued to collaborate with international companies to develop innovative speech intelligence and speech pathology products.