Abstract
Effective modeling of electronic health records (EHR) is rapidly becoming animportant topic in both academia and industry. A recent study showed thatutilizing the graphical structure underlying EHR data (e.g. relationshipbetween diagnoses and treatments) improves the performance of prediction taskssuch as heart failure diagnosis prediction. However, EHR data do not alwayscontain complete structure information. Moreover, when it comes to claims data,structure information is completely unavailable to begin with. Under suchcircumstances, can we still do better than just treating EHR data as aflat-structured bag-of-features? In this paper, we study the possibility ofutilizing the implicit structure of EHR by using the Transformer for predictiontasks on EHR data. Specifically, we argue that the Transformer is a suitablemodel to learn the hidden EHR structure, and propose the Graph ConvolutionalTransformer, which uses data statistics to guide the structure learningprocess. Our model empirically demonstrated superior prediction performance toprevious approaches on both synthetic data and publicly available EHR data onencounter-based prediction tasks such as graph reconstruction and readmissionprediction, indicating that it can serve as an effective general-purposerepresentation learning algorithm for EHR data.