Abstract
In order to alleviate data sparsity and overfitting problems in maximumlikelihood estimation (MLE) for sequence prediction tasks, we propose theGenerative Bridging Network (GBN), in which a novel bridge module is introducedto assist the training of the sequence prediction model (the generatornetwork). Unlike MLE directly maximizing the conditional likelihood, the bridgeextends the point-wise ground truth to a bridge distribution conditioned on it,and the generator is optimized to minimize their KL-divergence. Three differentGBNs, namely uniform GBN, language-model GBN and coaching GBN, are proposed topenalize confidence, enhance language smoothness and relieve learning burden.Experiments conducted on two recognized sequence prediction tasks (machinetranslation and abstractive text summarization) show that our proposed GBNs canyield significant improvements over strong baselines. Furthermore, by analyzingsamples drawn from different bridges, expected influences on the generator areverified.