Perfect machine translation (MT) would render cross-lingual transfer (XLT) bymeans of multilingual language models (LMs) superfluous. Given, on one hand,the large body of work on improving XLT with multilingual LMs and, on the otherhand, recent advances in massively multilingual MT, in this work, wesystematically evaluate existing and propose new translation-based XLTapproaches for transfer to low-resource languages. We show that alltranslation-based approaches dramatically outperform zero-shot XLT withmultilingual LMs, rendering the approach that combines the round-triptranslation of the source-language training data with the translation of thetarget-language test instances the most effective. We next show that one canobtain further empirical gains by adding reliable translations to otherhigh-resource languages to the training data. Moreover, we propose an effectivetranslation-based XLT strategy even for languages not supported by the MTsystem. Finally, we show that model selection for XLT based on target-languagevalidation data obtained with MT outperforms model selection based on thesource-language data. We hope that our findings encourage adoption of morerobust translation-based baselines in XLT research.