Abstract
Scientific extreme summarization (TLDR) aims to form ultra-short summaries ofscientific papers. Previous efforts on curating scientific TLDR datasets failedto scale up due to the heavy human annotation and domain expertise required. Inthis paper, we propose a simple yet effective approach to automaticallyextracting TLDR summaries for scientific papers from their citation texts.Based on the proposed approach, we create a new benchmark CiteSum without humanannotation, which is around 30 times larger than the previous human-curateddataset SciTLDR. We conduct a comprehensive analysis of CiteSum, examining itsdata characteristics and establishing strong baselines. We further demonstratethe usefulness of CiteSum by adapting models pre-trained on CiteSum (namedCITES) to new tasks and domains with limited supervision. For scientificextreme summarization, CITES outperforms most fully-supervised methods onSciTLDR without any fine-tuning and obtains state-of-the-art results with only128 examples. For news extreme summarization, CITES achieves significant gainson XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE-1zero-shot performance and state-of-the-art few-shot performance. For newsheadline generation, CITES performs the best among unsupervised and zero-shotmethods on Gigaword.