Glyce: Glyph-vectors for Chinese Character Representations

  • 2019-01-29 06:15:36
  • Wei Wu, Yuxian Meng, Qinghong Han, Muyu Li, Xiaoya Li, Jie Mei, Ping Nie, Xiaofei Sun, Jiwei Li
  • 8


It is intuitive that NLP tasks for logographic languages like Chinese shouldbenefit from the use of the glyph information in those languages. However, dueto the lack of rich pictographic evidence in glyphs and the weak generalizationability of standard computer vision models on character data, an effective wayto utilize the glyph information remains to be found. In this paper, we address this gap by presenting the Glyce, the glyph-vectorsfor Chinese character representations. We make three major innovations: (1) Weuse historical Chinese scripts (e.g., bronzeware script, seal script,traditional Chinese, etc) to enrich the pictographic evidence in characters;(2) We design CNN structures tailored to Chinese character image processing;and (3) We use image-classification as an auxiliary task in a multi-tasklearning setup to increase the model's ability to generalize. For the first time, we show that glyph-based models are able to consistentlyoutperform word/char ID-based models in a wide range of Chinese NLP tasks.Using Glyce, we are able to achieve the state-of-the-art performances on 13(almost all) Chinese NLP tasks, including (1) character-Level languagemodeling, (2) word-Level language modeling, (3) Chinese word segmentation, (4)name entity recognition, (5) part-of-speech tagging, (6) dependency parsing,(7) semantic role labeling, (8) sentence semantic similarity, (9) sentenceintention identification, (10) Chinese-English machine translation, (11)sentiment analysis, (12) document classification and (13) discourse parsing


Introduction (beta)



Conclusion (beta)