Abstract
Injecting external knowledge can improve the performance of pre-trainedlanguage models (PLMs) on various downstream NLP tasks. However, massiveretraining is required to deploy new knowledge injection methods or knowledgebases for downstream tasks. In this work, we are the first to study how toimprove the flexibility and efficiency of knowledge injection by reusingexisting downstream models. To this end, we explore a new paradigmplug-and-play knowledge injection, where knowledge bases are injected intofrozen existing downstream models by a knowledge plugin. Correspondingly, wepropose a plug-and-play injection method map-tuning, which trains a mapping ofknowledge embeddings to enrich model inputs with mapped embeddings whilekeeping model parameters frozen. Experimental results on three knowledge-drivenNLP tasks show that existing injection methods are not suitable for the newparadigm, while map-tuning effectively improves the performance of downstreammodels. Moreover, we show that a frozen downstream model can be well adapted todifferent domains with different mapping networks of domain knowledge. Our codeand models are available at https://github.com/THUNLP/Knowledge-Plugin.