Abstract
Despite advances in language modelling, distributional methods that buildsemantic representations from co-occurrences fail to discriminate betweenplausible and implausible events. In this work, we investigate how plausibilityprediction can be improved by injecting latent knowledge prompted from largelanguage models using parameter-efficient fine-tuning. We train 12 taskadapters to learn various physical properties and association measures andperform adapter fusion to compose latent semantic knowledge from each task ontop of pre-trained AlBERT embeddings. We automate auxiliary task datageneration, which enables us to scale our approach and fine-tune our learnedrepresentations across two plausibility datasets. Our code is available athttps://github.com/Jacob-Chmura/plausibility-vaccine.