• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Wang, Jianwen (Wang, Jianwen.) [1] | Wang, Shiping (Wang, Shiping.) [2] (Scholars:王石平) | Xiao, Shunxin (Xiao, Shunxin.) [3] | Lin, Renjie (Lin, Renjie.) [4] | Dong, Mianxiong (Dong, Mianxiong.) [5] | Guo, Wenzhong (Guo, Wenzhong.) [6] (Scholars:郭文忠)

Indexed by:

EI Scopus SCIE

Abstract:

Attention mechanism has been a successful method for multimodal affective analysis in recent years. Despite the advances, several significant challenges remain in fusing language and its nonverbal context information. One is to generate sparse attention coefficients associated with acoustic and visual modalities, which helps locate critical emotional semantics. The other is fusing complementary cross-modal representation to construct optimal salient feature combinations of multiple modalities. A Conditional Transformer Fusion Network is proposed to handle these problems. Firstly, the authors equip the transformer module with CNN layers to enhance the detection of subtle signal patterns in nonverbal sequences. Secondly, sentiment words are utilised as context conditions to guide the computation of cross-modal attention. As a result, the located nonverbal features are not only salient but also complementary to sentiment words directly. Experimental results show that the authors' method achieves state-of-the-art performance on several multimodal affective analysis datasets.

Keyword:

affective computing data fusion information fusion multimodal approaches

Community:

  • [ 1 ] [Wang, Jianwen]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 2 ] [Wang, Shiping]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 3 ] [Xiao, Shunxin]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 4 ] [Lin, Renjie]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 5 ] [Guo, Wenzhong]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 6 ] [Wang, Jianwen]Fujian Normal Univ, Coll Comp & Cyber Secur, Fuzhou, Peoples R China
  • [ 7 ] [Wang, Jianwen]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China
  • [ 8 ] [Wang, Shiping]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China
  • [ 9 ] [Xiao, Shunxin]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China
  • [ 10 ] [Lin, Renjie]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China
  • [ 11 ] [Guo, Wenzhong]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China
  • [ 12 ] [Wang, Jianwen]Fujian Normal Univ, Digital Fujian Inst Big Data Secur Technol, Fuzhou, Peoples R China
  • [ 13 ] [Dong, Mianxiong]Muroran Inst Technol, Dept Sci & Informat, Muroran, Japan

Reprint 's Address:

  • 郭文忠

    [Guo, Wenzhong]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China;;[Guo, Wenzhong]Fuzhou Univ, Key Lab Network Comp & Intelligent Informat Proc, Fuzhou, Peoples R China

Show more details

Related Keywords:

Related Article:

Source :

CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY

ISSN: 2468-6557

Year: 2024

Issue: 4

Volume: 9

Page: 917-931

8 . 4 0 0

JCR@2023

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count: 1

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Online/Total:120/10133507
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1