Abstract
Link Prediction(LP) is an essential task over Knowledge Graphs(KGs),traditionally focussed on using and predicting the relations between entities.Textual entity descriptions have already been shown to be valuable, but modelsthat incorporate numerical literals have shown minor improvements on existingbenchmark datasets. It is unclear whether a model is actually better in usingnumerical literals, or better capable of utilizing the graph structure. Thisraises doubts about the effectiveness of these methods and about thesuitability of the existing benchmark datasets. We propose a methodology to evaluate LP models that incorporate numericalliterals. We propose i) a new synthetic dataset to better understand how wellthese models use numerical literals and ii) dataset ablations strategies toinvestigate potential difficulties with the existing datasets. We identify aprevalent trend: many models underutilize literal information and potentiallyrely on additional parameters for performance gains. Our investigationhighlights the need for more extensive evaluations when releasing new modelsand datasets.