Abstract
Sequence to Sequence models struggle at compositionality and systematicgeneralisation even while they excel at many other tasks. We attribute thislimitation to their failure to internalise constructions conventionalised formmeaning pairings that license productive recombination. Building on theseinsights, we introduce an unsupervised procedure for miningpseudo-constructions: variable-slot templates automatically extracted fromtraining data. When applied to the SCAN dataset, our method yields large gainsout-of-distribution splits: accuracy rises to 47.8 %on ADD JUMP and to 20.3% onAROUND RIGHT without any architectural changes or additional supervision. Themodel also attains competitive performance with? 40% of the original trainingdata, demonstrating strong data efAciency. Our findings highlight the promiseof construction-aware preprocessing as an alternative to heavy architectural ortraining-regime interventions.