Abstract

We propose BERTScore, an automatic evaluation metric for text generation.Analogously to common metrics, BERTScore computes a similarity score for eachtoken in the candidate sentence with each token in the reference sentence.However, instead of exact matches, we compute token similarity using contextualembeddings. We evaluate using the outputs of 363 machine translation and imagecaptioning systems. BERTScore correlates better with human judgments andprovides stronger model selection performance than existing metrics. Finally,we use an adversarial paraphrase detection task to show that BERTScore is morerobust to challenging examples when compared to existing metrics.

 

Quick Read (beta)