Abstract
Embedding methods which enforce a partial order or lattice structure over theconcept space, such as Order Embeddings (OE) (Vendrov et al., 2016), are anatural way to model transitive relational data (e.g. entailment graphs).However, OE learns a deterministic knowledge base, limiting expressiveness ofqueries and the ability to use uncertainty for both prediction and learning(e.g. learning from expectations). Probabilistic extensions of OE (Lai andHockenmaier, 2017) have provided the ability to somewhat calibrate thesedenotational probabilities while retaining the consistency and inductive biasof ordered models, but lack the ability to model the negative correlationsfound in real-world knowledge. In this work we show that a broad class ofmodels that assign probability measures to OE can never capture negativecorrelation, which motivates our construction of a novel box lattice andaccompanying probability measure to capture anticorrelation and even disjointconcepts, while still providing the benefits of probabilistic modeling, such asthe ability to perform rich joint and conditional queries over arbitrary setsof concepts, and both learning from and predicting calibrated uncertainty. Weshow improvements over previous approaches in modeling the Flickr and WordNetentailment graphs, and investigate the power of the model.