Abstract

Measuring semantic change has thus far remained a task where methods usingcontextual embeddings have struggled to improve upon simpler techniques relyingonly on static word vectors. Moreover, many of the previously proposedapproaches suffer from downsides related to scalability and ease ofinterpretation. We present a simplified approach to measuring semantic changeusing contextual embeddings, relying only on the most probable substitutes formasked terms. Not only is this approach directly interpretable, it is also farmore efficient in terms of storage, achieves superior average performanceacross the most frequently cited datasets for this task, and allows for morenuanced investigation of change than is possible with static word vectors.

 

Quick Read (beta)