Abstract
State-of-the-art neural (re)rankers are notoriously data-hungry which --given the lack of large-scale training data in languages other than English --makes them rarely used in multilingual and cross-lingual retrieval settings.Current approaches therefore commonly transfer rankers trained on English datato other languages and cross-lingual setups by means of multilingual encoders:they fine-tune all parameters of pretrained massively multilingual Transformers(MMTs, e.g., multilingual BERT) on English relevance judgments, and then deploythem in the target language(s). In this work, we show that twoparameter-efficient approaches to cross-lingual transfer, namely SparseFine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and moreeffective zero-shot transfer to multilingual and cross-lingual retrieval tasks.We first train language adapters (or SFTMs) via Masked Language Modelling andthen train retrieval (i.e., reranking) adapters (SFTMs) on top, while keepingall other parameters fixed. At inference, this modular design allows us tocompose the ranker by applying the (re)ranking adapter (or SFTM) trained withsource language data together with the language adapter (or SFTM) of a targetlanguage. We carry out a large scale evaluation on the CLEF-2003 and HC4benchmarks and additionally, as another contribution, extend the former withqueries in three new languages: Kyrgyz, Uyghur and Turkish. The proposedparameter-efficient methods outperform standard zero-shot transfer with fullMMT fine-tuning, while being more modular and reducing training times. Thegains are particularly pronounced for low-resource languages, where ourapproaches also substantially outperform the competitive machinetranslation-based rankers.