Abstract
Cross-lingual knowledge transfer, especially between high- and low-resourcelanguages, remains a challenge in natural language processing (NLP). This studyoffers insights for improving cross-lingual NLP applications through thecombination of parameter-efficient fine-tuning methods. We systematicallyexplore strategies for enhancing this cross-lingual transfer through theincorporation of language-specific and task-specific adapters and soft prompts.We present a detailed investigation of various combinations of these methods,exploring their efficiency across six languages, focusing on three low-resourcelanguages, including the to our knowledge first use of soft language prompts.Our findings demonstrate that in contrast to claims of previous work, acombination of language and task adapters does not always work best; instead,combining a soft language prompt with a task adapter outperforms otherconfigurations in many cases.