Abstract

We introduce SentEval, a toolkit for evaluating the quality of universalsentence representations. SentEval encompasses a variety of tasks, includingbinary and multi-class classification, natural language inference and sentencesimilarity. The set of tasks was selected based on what appears to be thecommunity consensus regarding the appropriate evaluations for universalsentence representations. The toolkit comes with scripts to download andpreprocess datasets, and an easy interface to evaluate sentence encoders. Theaim is to provide a fairer, less cumbersome and more centralized way forevaluating sentence representations.

 

Quick Read (beta)