Abstract
Although there is an emerging trend towards generating embeddings forprimarily unstructured data, and recently for structured data, there is not yetany systematic suite for measuring the quality of embeddings. This deficiencyis further sensed with respect to embeddings generated for structured databecause there are no concrete evaluation metrics measuring the quality ofencoded structure as well as semantic patterns in the embedding space. In thispaper, we introduce a framework containing three distinct tasks concerned withthe individual aspects of ontological concepts: (i) the categorization aspect,(ii) the hierarchical aspect, and (iii) the relational aspect. Then, in thescope of each task, a number of intrinsic metrics are proposed for evaluatingthe quality of the embeddings. Furthermore, w.r.t. this framework multipleexperimental studies were run to compare the quality of the available embeddingmodels. Employing this framework in future research can reduce misjudgment andprovide greater insight about quality comparisons of embeddings for ontologicalconcepts.