Pieces of Eight: 8-bit Neural Machine Translation

  • 2018-04-13 17:10:12
  • Jerry Quinn, Miguel Ballesteros
  • 1


Neural machine translation has achieved levels of fluency and adequacy thatwould have been surprising a short time ago. Output quality is extremelyrelevant for industry purposes, however it is equally important to produceresults in the shortest time possible, mainly for latency-sensitiveapplications and to control cloud hosting costs. In this paper we show theeffectiveness of translating with 8-bit quantization for models that have beentrained using 32-bit floating point values. Results show that 8-bit translationmakes a non-negligible impact in terms of speed with no degradation in accuracyand adequacy.


Introduction (beta)



Conclusion (beta)