| DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection | |
| Y. Tao; M. Yang; H. Li; Y. Wu; B. Hu | |
| 2024-01-05 | |
| Source Publication | IEEE Transactions on Knowledge and Data Engineering Impact Factor & Quartile Of Published Year The Latest Impact Factor & Quartile |
| ISSN | 1558-2191 |
| EISSN | 1558-2191 |
| Volume | PPIssue:99Pages:1-12 |
| page numbers | 11 |
| Abstract | Depression is one of the most common mental illnesses, but few of the currently proposed in-depth models based on social media data take into account both temporal and spatial information in the data for the detection of depression. In this paper, we present an efficient, low-covariance multimodal integrated spatio-temporal converter framework called DepMSTAT, which aims to detect depression using acoustic and visual features in social media data. The framework consists of four modules: a data preprocessing module, a token generation module, a Spatial-Temporal Attentional Transformer (STAT) module, and a depression classifier module. To efficiently capture spatial and temporal correlations in multimodal social media depression data, a plug-and-play STAT module is proposed. The module is capable of extracting unimodal spatio-temporal features and fusing unimodal information, playing a key role in the analysis of acoustic and visual features in social media data. Through extensive experiments on a depression database (D-Vlog), the method in this paper shows high accuracy (71.53%) in depression detection, achieving a performance that exceeds most models. This work provides a scaffold for studies based on multimodal data that assists in the detection of depression. |
| Keyword | Depression detection Transformer Spatio-temporal attention Vlog data |
| Publisher | IEEE |
| DOI | 10.1109/TKDE.2024.3350071 |
| Indexed By | IEEE ; SCIE |
| Language | 英语 |
| WOS Research Area | Computer Science ; Engineering |
| WOS Subject | Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical & Electronic |
| WOS ID | WOS:001245017200014 |
| Original Document Type | Article |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | https://ir.lzu.edu.cn/handle/262010/582874 |
| Collection | 信息科学与工程学院 |
| Affiliation | 1.Gansu Provincial Key Laboratory of Wearable Computing School of information Science and Engineering, Lanzhou University, LanZhou, China 2.Gansu Provincial Key Laboratory of Wearable Computing School of information Science and Engineering, Lanzhou University, LanZhou, China 3.Gansu Provincial Key Laboratory of Wearable Computing School of information Science and Engineering, Lanzhou University, LanZhou, China 4.Gansu Provincial Key Laboratory of Wearable Computing School of information Science and Engineering, Lanzhou University, LanZhou, China 5.Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou, China |
| Recommended Citation GB/T 7714 | Y. Tao,M. Yang,H. Li,et al. DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection[J]. IEEE Transactions on Knowledge and Data Engineering,2024,PP(99):1-12. |
| APA | Y. Tao,M. Yang,H. Li,Y. Wu,&B. Hu.(2024).DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection.IEEE Transactions on Knowledge and Data Engineering,PP(99),1-12. |
| MLA | Y. Tao,et al."DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection".IEEE Transactions on Knowledge and Data Engineering PP.99(2024):1-12. |
| Files in This Item: | There are no files associated with this item. | ||||
