Abstract
Dynamic treatment recommendation systems based on large-scale electronichealth records (EHRs) become a key to successfully improve practical clinicaloutcomes. Prior relevant studies recommend treatments either use supervisedlearning (e.g. matching the indicator signal which denotes doctorprescriptions), or reinforcement learning (e.g. maximizing evaluation signalwhich indicates cumulative reward from survival rates). However, none of thesestudies have considered to combine the benefits of supervised learning andreinforcement learning. In this paper, we propose Supervised ReinforcementLearning with Recurrent Neural Network (SRL-RNN), which fuses them into asynergistic learning framework. Specifically, SRL-RNN applies an off-policyactor-critic framework to handle complex relations among multiple medications,diseases and individual characteristics. The "actor" in the framework isadjusted by both the indicator signal and evaluation signal to ensure effectiveprescription and low mortality. RNN is further utilized to solve thePartially-Observed Markov Decision Process (POMDP) problem due to the lack offully observed states in real world applications. Experiments on the publiclyreal-world dataset, i.e., MIMIC-3, illustrate that our model can reduce theestimated mortality, while providing promising accuracy in matching doctors'prescriptions.