Abstract
The retrieval model is an indispensable component for real-worldknowledge-intensive tasks, e.g., open-domain question answering (ODQA). Asseparate retrieval skills are annotated for different datasets, recent workfocuses on customized methods, limiting the model transferability andscalability. In this work, we propose a modular retriever where individualmodules correspond to key skills that can be reused across datasets. Ourapproach supports flexible skill configurations based on the target domain toboost performance. To mitigate task interference, we design a novelmodularization parameterization inspired by sparse Transformer. We demonstratethat our model can benefit from self-supervised pretraining on Wikipedia andfine-tuning using multiple ODQA datasets, both in a multi-task fashion. Ourapproach outperforms recent self-supervised retrievers in zero-shot evaluationsand achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQAand OTT-QA.