In this paper, we propose a novel hierarchical dirichlet process-basedclassification framework for open set recognition (HDP-OSR) where newcategories' samples unseen in training appear during testing. Unlike theexisting methods which deal with this problem from the perspective ofdiscriminative model, we reconsider this problem from the perspective ofgenerative model. We model each known class data in training set as a group inhierarchical dirichlet process (HDP) while the testing set as a whole istreated in the same way, then co-clustering all the groups under the HDPframework. Based on the properties of HDP, our HDP-OSR does not overly dependon training samples and can achieve adaptive change as the data changes. Moreprecisely, HDP-OSR can automatically reserve space for unknown categories whileit can also discover new categories, meaning it naturally adapts to the openset recognition scenario. Furthermore, treating the testing set as a wholemakes our framework take the correlations among the testing samples intoaccount whereas the existing methods obviously ignore this information.Experimental results on a set of benchmark data sets indicate the validity ofour learning framework.