Word2Vec applied to Recommendation: Hyperparameters Matter

  • 2018-04-11 20:37:35
  • Hugo Caselles-Dupré, Florian Lesaint, Jimena Royo-Letelier
  • 48

Abstract

Skip-gram with negative sampling, a popular variant of Word2vec originallydesigned and tuned to create word embeddings for Natural Language Processing,has been used to create item embeddings with successful applications inrecommendation. While these fields do not share the same type of data, neitherevaluate on the same tasks, recommendation applications tend to use the samealready tuned hyperparameters values, even if optimal hyperparameters valuesare often known to be data and task dependent. We thus investigate the marginalimportance of each hyperparameter in a recommendation setting, with anextensive joint hyperparameter optimization on various datasets. Results revealthat optimizing neglected hyperparameters, namely negative samplingdistribution, number of epochs, subsampling parameter and window-size,significantly improves performance on a recommendation task, and can increaseit up to a factor of $10$.

 

Quick Read (beta)

loading the full paper ...