Abstract
Recently, we have witnessed a rise in the use of Large Language Models(LLMs), especially in applications like chatbot assistants. Safety mechanismsand specialized training procedures are implemented to prevent improperresponses from these assistants. In this work, we bypass these measures forChatGPT and Gemini (and, to some extent, Bing chat) by making them impersonatecomplex personas with personality characteristics that are not aligned with atruthful assistant. We start by creating elaborate biographies of thesepersonas, which we then use in a new session with the same chatbots. Ourconversations then follow a role-play style to elicit prohibited responses.Using personas, we show that prohibited responses are actually provided, makingit possible to obtain unauthorized, illegal, or harmful information. This workshows that by using adversarial personas, one can overcome safety mechanismsset out by ChatGPT and Gemini. We also introduce several ways of activatingsuch adversarial personas, which show that both chatbots are vulnerable to thiskind of attack. With the same principle, we introduce two defenses that pushthe model to interpret trustworthy personalities and make it more robustagainst such attacks.