Abstract
Translationese refers to linguistic properties that usually occur intranslated texts. Previous works study translationese by framing it as a binaryclassification between original texts and translated texts. In this paper, weargue that translationese should be graded instead of binary and propose thefirst measure for translationese -- the translationese-index (T-index),computed from the likelihood ratios of two contrastively fine-tuned languagemodels (LMs). We use synthesized translations and translations in the wild toevaluate T-index's generalizability in cross-domain settings and its validityagainst human judgments. Our results show that T-index can generalize to unseengenres, authors, and language pairs. Moreover, T-index computed using two 0.5BLMs fine-tuned on only 1-5k pairs of synthetic data can effectively capturetranslationese, as demonstrated by alignment with human pointwise ratings andpairwise judgments. Additionally, the correlation between T-index and existingmachine translation (MT) quality estimation (QE) metrics such as BLEU and COMETis low, suggesting that T-index is not covered by these metrics and can serveas a complementary metric in MT QE.