Abstract
Deep learning has recently demonstrated state-of-the art performance on keytasks related to the maintenance of computer systems, such as intrusiondetection, denial of service attack detection, hardware and software systemfailures, and malware detection. In these contexts, model interpretability isvital for administrator and analyst to trust and act on the automated analysisof machine learning models. Deep learning methods have been criticized as blackbox oracles which allow limited insight into decision factors. In this work weseek to "bridge the gap" between the impressive performance of deep learningmodels and the need for interpretable model introspection. To this end wepresent recurrent neural network (RNN) language models augmented with attentionfor anomaly detection in system logs. Our methods are generally applicable toany computer system and logging source. By incorporating attention variants into our RNN language models we createopportunities for model introspection and analysis without sacrificingstate-of-the art performance. We demonstrate model performance and illustrate model interpretability on anintrusion detection task using the Los Alamos National Laboratory (LANL) cybersecurity dataset, reporting upward of 0.99 area under the receiver operatorcharacteristic curve despite being trained only on a single day's worth ofdata.