Electronic health records (EHR) are increasingly being used for constructingdisease risk prediction models. Feature engineering in EHR data however ischallenging due to their highly dimensional and heterogeneous nature.Low-dimensional representations of EHR data can potentially mitigate thesechallenges. In this paper, we use global vectors (GloVe) to learn wordembeddings for diagnoses and procedures recorded using 13 million ontologyterms across 2.7 million hospitalisations in national UK EHR. We demonstratethe utility of these embeddings by evaluating their performance in identifyingpatients which are at higher risk of being hospitalised for congestive heartfailure. Our findings indicate that embeddings can enable the creation ofrobust EHR-derived disease risk prediction models and address some thelimitations associated with manual clinical feature engineering.