Abstract

Neural machine translation has achieved levels of fluency and adequacy thatwould have been surprising a short time ago. Output quality is extremelyrelevant for industry purposes, however it is equally important to produceresults in the shortest time possible, mainly for latency-sensitiveapplications and to control cloud hosting costs. In this paper we show theeffectiveness of translating with 8-bit quantization for models that have beentrained using 32-bit floating point values. Results show that 8-bit translationmakes a non-negligible impact in terms of speed with no degradation in accuracyand adequacy.

 

Quick Read (beta)