Attention-based Graph Neural Network for Semi-supervised Learning

  • 2018-03-10 02:01:35
  • Kiran K. Thekumparampil, Chong Wang, Sewoong Oh, Li-Jia Li
  • 26

Abstract

Recently popularized graph neural networks achieve the state-of-the-artaccuracy on a number of standard benchmark datasets for graph-basedsemi-supervised learning, improving significantly over existing approaches.These architectures alternate between a propagation layer that aggregates thehidden states of the local neighborhood and a fully-connected layer. Perhapssurprisingly, we show that a linear model, that removes all the intermediatefully-connected layers, is still able to achieve a performance comparable tothe state-of-the-art models. This significantly reduces the number ofparameters, which is critical for semi-supervised learning where number oflabeled examples are small. This in turn allows a room for designing moreinnovative propagation layers. Based on this insight, we propose a novel graphneural network that removes all the intermediate fully-connected layers, andreplaces the propagation layers with attention mechanisms that respect thestructure of the graph. The attention mechanism allows us to learn a dynamicand adaptive local summary of the neighborhood to achieve more accuratepredictions. In a number of experiments on benchmark citation networksdatasets, we demonstrate that our approach outperforms competing methods. Byexamining the attention weights among neighbors, we show that our modelprovides some interesting insights on how neighbors influence each other.

 

Quick Read (beta)

loading the full paper ...