Humor is an essential human trait. Efforts to understand humor have calledout links between humor and the foundations of cognition, as well as theimportance of humor in social engagement. As such, it is a promising andimportant subject of study, with relevance for artificial intelligence andhuman-computer interaction. Previous computational work on humor has mostlyoperated at a coarse level of granularity, e.g., predicting whether an entiresentence, paragraph, document, etc., is humorous. As a step toward deepunderstanding of humor, we seek fine-grained models of attributes that make agiven text humorous. Starting from the observation that satirical newsheadlines tend to resemble serious news headlines, we build and analyze acorpus of satirical headlines paired with nearly identical but seriousheadlines. The corpus is constructed via Unfun.me, an online game thatincentivizes players to make minimal edits to satirical headlines with the goalof making other players believe the results are serious headlines. The editoperations used to successfully remove humor pinpoint the words and conceptsthat play a key role in making the original, satirical headline funny. Ouranalysis reveals that the humor tends to reside toward the end of headlines,and primarily in noun phrases, and that most satirical headlines follow acertain logical pattern, which we term false analogy. Overall, this paperdeepens our understanding of the syntactic and semantic structure of satiricalnews headlines and provides insights for building humor-producing systems.