Abstract
In parallel to their overwhelming success across NLP tasks, language abilityof deep Transformer networks, pretrained via language modeling (LM) objectiveshas undergone extensive scrutiny. While probing revealed that these modelsencode a range of syntactic and semantic properties of a language, they arestill prone to fall back on superficial cues and simple heuristics to solvedownstream tasks, rather than leverage deeper linguistic knowledge. In thispaper, we target one such area of their deficiency, verbal reasoning. Weinvestigate whether injecting explicit information on verbs' semantic-syntacticbehaviour improves the performance of LM-pretrained Transformers in eventextraction tasks -- downstream tasks for which accurate verb processing isparamount. Concretely, we impart the verb knowledge from curated lexicalresources into dedicated adapter modules (dubbed verb adapters), allowing it tocomplement, in downstream tasks, the language knowledge obtained duringLM-pretraining. We first demonstrate that injecting verb knowledge leads toperformance gains in English event extraction. We then explore the utility ofverb adapters for event extraction in other languages: we investigate (1)zero-shot language transfer with multilingual Transformers as well as (2)transfer via (noisy automatic) translation of English verb-based lexicalconstraints. Our results show that the benefits of verb knowledge injectionindeed extend to other languages, even when verb adapters are trained onnoisily translated constraints.