| Dynamic Alignment and Fusion of Multimodal Physiological Patterns for Stress Recognition | |
| X. Zhang; X. Wei; Z. Zhou; Q. Zhao; S. Zhang; Y. Yang; R. Li; B. Hu | |
| 2023-06-28 | |
| Source Publication | IEEE Transactions on Affective Computing Impact Factor & Quartile |
| ISSN | 1949-3045 |
| Volume | PPIssue:99Pages:1-12 |
| Abstract | Stress has been identified as one of major causes of health issues. To detect the stress levels with higher accuracy, fusion of multimodal physiological signals is a promising technique. However, there is an asynchrony between physiological signals observed from different perspectives. Exploring the temporal alignment relationship between modalities is helpful to improve the quality of multimodal fusion. This paper proposes an end-to-end multimodal stress detection model based on Bidirectional Cross- and Self-modal Attention (BCSA) mechanism. Specifically, we first construct different feature extractors based on the characteristics of Blood Volume Pulse (BVP) and Electrodermal Activity (EDA) to complete automated temporal feature extraction. Secondly, cross-modal attention is used to seek the alignment relationship between the two modalities and fully fuse cross-modal information. The self-modal attention is used to attenuate noise and redundant information, highlight important information and obtain salient stress representations. Finally, the stress representations of the two modalities are processed separately, and the mean square error (MSE) is used to narrow the gap between them. Experimental results on the UBFC-Phys dataset and WESAD dataset show that the proposed model can effectively improve the accuracy of stress recognition, and outperforms several state-of-the-art methods. |
| Keyword | Stress Recognition Physiological Signal Multimodal Fusion Self Attention Cross Attention |
| Publisher | IEEE |
| DOI | 10.1109/TAFFC.2023.3290177 |
| Indexed By | IEEE |
| Language | 英语 |
| EI Accession Number | 20232714335243 |
| EI Keywords | Physiology |
| EI Classification Number | 461.4 Ergonomics and Human Factors Engineering ; 461.9 Biology ; 716.1 Information Theory and Signal Processing ; 802.3 Chemical Operations ; 922.2 Mathematical Statistics |
| Original Document Type | Article in Press |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | https://ir.lzu.edu.cn/handle/262010/529075 |
| Collection | 兰州大学 信息科学与工程学院 |
| Affiliation | 1.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 2.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 3.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 4.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 5.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 6.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 7.Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering, Lanzhou University, Lanzhou, China 8.School of Medical Technology, Beijing Institute of Technology, Beijing, China |
| Recommended Citation GB/T 7714 | X. Zhang,X. Wei,Z. Zhou,et al. Dynamic Alignment and Fusion of Multimodal Physiological Patterns for Stress Recognition[J]. IEEE Transactions on Affective Computing,2023,PP(99):1-12. |
| APA | X. Zhang.,X. Wei.,Z. Zhou.,Q. Zhao.,S. Zhang.,...&B. Hu.(2023).Dynamic Alignment and Fusion of Multimodal Physiological Patterns for Stress Recognition.IEEE Transactions on Affective Computing,PP(99),1-12. |
| MLA | X. Zhang,et al."Dynamic Alignment and Fusion of Multimodal Physiological Patterns for Stress Recognition".IEEE Transactions on Affective Computing PP.99(2023):1-12. |
| Files in This Item: | There are no files associated with this item. | ||||
