Abstract
Abstractive summarization systems today produce fluent and relevant output,but often "hallucinate" statements not supported by the source text. We analyzethe connection between hallucinations and training data, and find evidence thatmodels hallucinate because they train on target summaries that are unsupportedby the source. Based on our findings, we present PINOCCHIO, a new decodingmethod that improves the consistency of a transformer-based abstractivesummarizer by constraining beam search to avoid hallucinations. Given the modelstates and outputs at a given step, PINOCCHIO detects likely modelhallucinations based on various measures of attribution to the source text.PINOCCHIO backtracks to find more consistent output, and can opt to produce nosummary at all when no consistent generation can be found. In experiments, wefind that PINOCCHIO improves the consistency of generation (in terms of F1) byan average of~67% on two abstractive summarization datasets.