DualKanbaFormer: Kolmogorov-Arnold Networks and State Space Model Transformer for Multimodal Aspect-based Sentiment Analysis

  • 2024-08-30 17:30:39
  • Adamu Lawan, Juhua Pu, Haruna Yunusa, Muhammad Lawan, Aliyu Umar, Adamu Sani Yahya
  • 0

Abstract

Multimodal aspect-based sentiment analysis (MABSA) enhances sentimentdetection by combining text with other data types like images. However, despitesetting significant benchmarks, attention mechanisms exhibit limitations inefficiently modelling long-range dependencies between aspect and opiniontargets within the text. They also face challenges in capturing global-contextdependencies for visual representations. To this end, we proposeKolmogorov-Arnold Networks (KANs) and Selective State Space model (Mamba)transformer (DualKanbaFormer), a novel architecture to address the aboveissues. We leverage the power of Mamba to capture global context dependencies,Multi-head Attention (MHA) to capture local context dependencies, and KANs tocapture non-linear modelling patterns for both textual representations (textualKanbaFormer) and visual representations (visual KanbaFormer). Furthermore, wefuse the textual KanbaFormer and visual KanbaFomer with a gated fusion layer tocapture the inter-modality dynamics. According to extensive experimentalresults, our model outperforms some state-of-the-art (SOTA) studies on twopublic datasets.

 

Quick Read (beta)

loading the full paper ...