Abstract
Anomaly detection (AD) in a surveillance scenario is an emerging andchallenging field of research. For autonomous vehicles like drones or cars, itis immensely important to distinguish between normal and abnormal states inreal-time. Additionally, we also need to detect any device malfunction. But thenature and degree of abnormality may vary depending upon the actual environmentand adversary. As a result, it is impractical to model all cases a-priori anduse supervised methods to classify. Also, an autonomous vehicle providesvarious data types like images and other analog or digital sensor data, all ofwhich can be useful in anomaly detection if leveraged fruitfully. To thateffect, in this paper, a heterogeneous system is proposed which estimates thedegree of abnormality of an unmanned surveillance drone, analyzing real-timeimage and IMU (Inertial Measurement Unit) sensor data in an unsupervisedmanner. Here, we have demonstrated a Convolutional Neural Network (CNN)architecture, named AngleNet to estimate the angle between a normal image andanother image under consideration, which provides us with a measure of anomalyof the device. Moreover, the IMU data are used in autoencoder to predictabnormality. Finally, the results from these two algorithms are ensembled toestimate the final degree of abnormality. The proposed method performssatisfactorily on the IEEE SP Cup-2020 dataset with an accuracy of 97.3%.Additionally, we have also tested this approach on an in-house dataset tovalidate its robustness.