An Analysis on the Learning Rules of the Skip-Gram Model

  • 2020-03-18 22:17:48
  • Canlin Zhang, Xiuwen Liu, Daniel Bis
  • 21

Abstract

To improve the generalization of the representations for natural languageprocessing tasks, words are commonly represented using vectors, where distancesamong the vectors are related to the similarity of the words. While word2vec,the state-of-the-art implementation of the skip-gram model, is widely used andimproves the performance of many natural language processing tasks, itsmechanism is not yet well understood. In this work, we derive the learning rules for the skip-gram model andestablish their close relationship to competitive learning. In addition, weprovide the global optimal solution constraints for the skip-gram model andvalidate them by experimental results.

 

Quick Read (beta)

loading the full paper ...