Abstract
Human activity recognition (HAR) on smartglasses has various use cases,including health/fitness tracking and input for context-aware AI assistants.However, current approaches for egocentric activity recognition suffer from lowperformance or are resource-intensive. In this work, we introduce a resource(memory, compute, power, sample) efficient machine learning algorithm,EgoCHARM, for recognizing both high level and low level activities using asingle egocentric (head-mounted) Inertial Measurement Unit (IMU). Ourhierarchical algorithm employs a semi-supervised learning strategy, requiringprimarily high level activity labels for training, to learn generalizable lowlevel motion embeddings that can be effectively utilized for low level activityrecognition. We evaluate our method on 9 high level and 3 low level activitiesachieving 0.826 and 0.855 F1 scores on high level and low level activityrecognition respectively, with just 63k high level and 22k low level modelparameters, allowing the low level encoder to be deployed directly on currentIMU chips with compute. Lastly, we present results and insights from asensitivity analysis and highlight the opportunities and limitations ofactivity recognition using egocentric IMUs.