Abstract
Multimodal wearable sensor data classification plays an important role inubiquitous computing and has a wide range of applications in scenarios fromhealthcare to entertainment. However, most existing work in this field employsdomain-specific approaches and is thus ineffective in complex sit- uationswhere multi-modality sensor data are col- lected. Moreover, the wearable sensordata are less informative than the conventional data such as texts or images.In this paper, to improve the adapt- ability of such classification methodsacross differ- ent application domains, we turn this classification task into agame and apply a deep reinforcement learning scheme to deal with complexsituations dynamically. Additionally, we introduce a selective attentionmechanism into the reinforcement learn- ing scheme to focus on the crucialdimensions of the data. This mechanism helps to capture extra information fromthe signal and thus it is able to significantly improve the discriminativepower of the classifier. We carry out several experiments on three wearablesensor datasets and demonstrate the competitive performance of the proposedapproach compared to several state-of-the-art baselines.