兰州大学机构库
Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism
Hou, Kechen1; Zhang, Xiaowei1; Yang, Yikun1; Zhao, Qiqi1; Yuan, Wenjie1; Zhou, Zhongyi1; Zhang, Sipo1; Li, Chen1; Shen, Jian2; Hu, Bin2
2023-10-20
Online publication date2023-10
Source PublicationIEEE Transactions on Cybernetics   Impact Factor & Quartile
ISSN2168-2267 ; 2168-2275
EISSN2168-2275
VolumePPIssue:99Pages:1-14
page numbers14
AbstractModeling correlations between multimodal physiological signals e.g., canonical correlation analysis (CCA) for emotion recognition has attracted much attention. However, existing studies rarely consider the neural nature of emotional responses within physiological signals. Furthermore, during fusion space construction, the CCA method maximizes only the correlations between different modalities and neglects the discriminative information of different emotional states. Most importantly, temporal mismatches between different neural activities are often ignored; therefore, the theoretical assumptions that multimodal data should be aligned in time and space before fusion are not fulfilled. To address these issues, we propose a discriminative correlation fusion method coupled with a temporal alignment mechanism for multimodal physiological signals. We first use neural signal analysis techniques to construct neural representations of the central nervous system (CNS) and autonomic nervous system (ANS). respectively. Then, emotion class labels are introduced in CCA to obtain more discriminative fusion representations from multimodal neural responses, and the temporal alignment between the CNS and ANS is jointly optimized with a fusion procedure that applies the Bayesian algorithm. The experimental results demonstrate that our method significantly improves the emotion recognition performance. Additionally, we show that this fusion method can model the underlying mechanisms in human nervous systems during emotional responses, and our results are consistent with prior findings. This study may guide a new approach for exploring human cognitive function based on physiological signals at different time scales and promote the development of computational intelligence and harmonious human-computer interactions.
KeywordCoordinated fusion emotion recognition nervous systems temporal alignment
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI10.1109/TCYB.2023.3320107
Indexed BySCIE ; IEEE
Language英语
WOS Research AreaAutomation & Control Systems ; Computer Science
WOS SubjectAutomation & Control Systems ; Computer Science, Artificial Intelligence ; Computer Science, Cybernetics
WOS IDWOS:001092397300001
Original Document TypeArticle ; Early Access
PMID 37862275
Citation statistics
Document Type期刊论文
Identifierhttps://ir.lzu.edu.cn/handle/262010/569288
Collection兰州大学
Corresponding AuthorZhang, Xiaowei; Hu, Bin
Affiliation
1.Lanzhou Univ, Sch Informat Sci & Engn, Gansu Prov Key Lab Wearable Comp, Lanzhou 730000, Peoples R China;
2.Beijing Inst Technol, Sch Med Technol, Beijing 10081, Peoples R China
Recommended Citation
GB/T 7714
Hou, Kechen,Zhang, Xiaowei,Yang, Yikun,et al. Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism[J]. IEEE Transactions on Cybernetics,2023,PP(99):1-14.
APA Hou, Kechen.,Zhang, Xiaowei.,Yang, Yikun.,Zhao, Qiqi.,Yuan, Wenjie.,...&Hu, Bin.(2023).Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism.IEEE Transactions on Cybernetics,PP(99),1-14.
MLA Hou, Kechen,et al."Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism".IEEE Transactions on Cybernetics PP.99(2023):1-14.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Altmetrics Score
Google Scholar
Similar articles in Google Scholar
[Hou, Kechen]'s Articles
[Zhang, Xiaowei]'s Articles
[Yang, Yikun]'s Articles
Baidu academic
Similar articles in Baidu academic
[Hou, Kechen]'s Articles
[Zhang, Xiaowei]'s Articles
[Yang, Yikun]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Hou, Kechen]'s Articles
[Zhang, Xiaowei]'s Articles
[Yang, Yikun]'s Articles
Terms of Use
No data!
Social Bookmark/Share
No comment.
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.