Abstract
We propose the Limited Multi-Label (LML) projection layer as a new primitiveoperation for end-to-end learning systems. The LML layer provides aprobabilistic way of modeling multi-label predictions limited to having exactlyk labels. We derive efficient forward and backward passes for this layer andshow how the layer can be used to optimize the top-k recall for multi-labeltasks with incomplete label information. We evaluate LML layers on top-kCIFAR-100 classification and scene graph generation. We show that LML layersadd a negligible amount of computational overhead, strictly improve the model'srepresentational capacity, and improve accuracy. We also revisit the truncatedtop-k entropy method as a competitive baseline for top-k classification.