Abstract
Fine-grained open-set recognition (FineOSR) aims to recognize imagesbelonging to classes with subtle appearance differences while rejecting imagesof unknown classes. A recent trend in OSR shows the benefit of generativemodels to discriminative unknown detection. As a type of generative model,energy-based models (EBM) are the potential for hybrid modeling of generativeand discriminative tasks. However, most existing EBMs suffer from densityestimation in high-dimensional space, which is critical to recognizing imagesfrom fine-grained classes. In this paper, we explore the low-dimensional latentspace with energy-based prior distribution for OSR in a fine-grained visualworld. Specifically, based on the latent space EBM, we propose anattribute-aware information bottleneck (AIB), a residual attribute featureaggregation (RAFA) module, and an uncertainty-based virtual outlier synthesis(UVOS) module to improve the expressivity, granularity, and density of thesamples in fine-grained classes, respectively. Our method is flexible to takeadvantage of recent vision transformers for powerful visual classification andgeneration. The method is validated on both fine-grained and general visualclassification datasets while preserving the capability of generatingphoto-realistic fake images with high resolution.