Abstract
Representation learning (RL) methods learn objects' latent embeddings whereinformation is preserved by distances. Since distances are invariant to certainlinear transformations, one may obtain different embeddings while preservingthe same information. In dynamic systems, a temporal difference in embeddingsmay be explained by the stability of the system or by the misalignment ofembeddings due to arbitrary transformations. In the literature, embeddingalignment has not been defined formally, explored theoretically, or analyzedempirically. Here, we explore the embedding alignment and its parts, providethe first formal definitions, propose novel metrics to measure alignment andstability, and show their suitability through synthetic experiments. Real-worldexperiments show that both static and dynamic RL methods are prone to producemisaligned embeddings and such misalignment worsens the performance of dynamicnetwork inference tasks. By ensuring alignment, the prediction accuracy raisesby up to 90% in static and by 40% in dynamic RL methods.