The ability to identify whether or not a test sample belongs to one of thesemantic classes in a classifier's training set is critical to practicaldeployment of the model. This task is termed open-set recognition (OSR) and hasreceived significant attention in recent years. In this paper, we firstdemonstrate that the ability of a classifier to make the 'none-of-above'decision is highly correlated with its accuracy on the closed-set classes. Wefind that this relationship holds across loss objectives and architectures, andfurther demonstrate the trend both on the standard OSR benchmarks as well as ona large-scale ImageNet evaluation. Second, we use this correlation to boost theperformance of the cross-entropy OSR 'baseline' by improving its closed-setaccuracy, and with this strong baseline achieve a new state-of-the-art on themost challenging OSR benchmark. Similarly, we boost the performance of theexisting state-of-the-art method by improving its closed-set accuracy, but thisdoes not surpass the strong baseline on the most challenging dataset. Our thirdcontribution is to reappraise the datasets used for OSR evaluation, andconstruct new benchmarks which better respect the task of detecting semanticnovelty, as opposed to low-level distributional shifts as tackled byneighbouring machine learning fields. In this new setting, we again demonstratethat there is negligible difference between the strong baseline and theexisting state-of-the-art.