Compressive summarization systems typically rely on a crafted set ofsyntactic rules to determine what spans of possible summary sentences can bedeleted, then learn a model of what to actually delete by optimizing forcontent selection (ROUGE). In this work, we propose to relax the rigidsyntactic constraints on candidate spans and instead leave compressiondecisions to two data-driven criteria: plausibility and salience. Deleting aspan is plausible if removing it maintains the grammaticality and factuality ofa sentence, and spans are salient if they contain important information fromthe summary. Each of these is judged by a pre-trained Transformer model, andonly deletions that are both plausible and not salient can be applied. Whenintegrated into a simple extraction-compression pipeline, our method achievesstrong in-domain results on benchmark summarization datasets, and humanevaluation shows that the plausibility model generally selects for grammaticaland factual deletions. Furthermore, the flexibility of our approach allows itto generalize cross-domain: our system fine-tuned on only 500 samples from anew domain can match or exceed an in-domain extractive model trained on muchmore data.