Abstract
In recent years, Large Language Models (LLM) have emerged as pivotal tools invarious applications. However, these models are susceptible to adversarialprompt attacks, where attackers can carefully curate input strings that misleadLLMs into generating incorrect or undesired outputs. Previous work has revealedthat with relatively simple yet effective attacks based on discreteoptimization, it is possible to generate adversarial prompts that bypassmoderation and alignment of the models. This vulnerability to adversarialprompts underscores a significant concern regarding the robustness andreliability of LLMs. Our work aims to address this concern by introducing anovel approach to detecting adversarial prompts at a token level, leveragingthe LLM's capability to predict the next token's probability. We measure thedegree of the model's perplexity, where tokens predicted with high probabilityare considered normal, and those exhibiting high perplexity are flagged asadversarial. Additionaly, our method also integrates context understanding byincorporating neighboring token information to encourage the detection ofcontiguous adversarial prompt sequences. To this end, we design two algorithmsfor adversarial prompt detection: one based on optimization techniques andanother on Probabilistic Graphical Models (PGM). Both methods are equipped withefficient solving methods, ensuring efficient adversarial prompt detection. Ourtoken-level detection result can be visualized as heatmap overlays on the textsequence, allowing for a clearer and more intuitive representation of whichpart of the text may contain adversarial prompts.