Abstract

Representations of sets are challenging to learn because operations on setsshould be permutation-invariant. To this end, we propose aPermutation-Optimisation module that learns how to permute a set end-to-end.The permuted set can be further processed to learn a permutation-invariantrepresentation of that set, avoiding a bottleneck in traditional set models. Wedemonstrate our model's ability to learn permutations and set representationswith either explicit or implicit supervision on four datasets, on which weachieve state-of-the-art results: number sorting, image mosaics, classificationfrom image mosaics, and visual question answering.

 

Quick Read (beta)