Abstract
Large Language Models (LLMs) have demonstrated impressive fluency andreasoning capabilities, but their potential for misuse has raised growingconcern. In this paper, we present ScamAgent, an autonomous multi-turn agentbuilt on top of LLMs, capable of generating highly realistic scam call scriptsthat simulate real-world fraud scenarios. Unlike prior work focused onsingle-shot prompt misuse, ScamAgent maintains dialogue memory, adaptsdynamically to simulated user responses, and employs deceptive persuasionstrategies across conversational turns. We show that current LLM safetyguardrails, including refusal mechanisms and content filters, are ineffectiveagainst such agent-based threats. Even models with strong prompt-levelsafeguards can be bypassed when prompts are decomposed, disguised, or deliveredincrementally within an agent framework. We further demonstrate thetransformation of scam scripts into lifelike voice calls using moderntext-to-speech systems, completing a fully automated scam pipeline. Ourfindings highlight an urgent need for multi-turn safety auditing, agent-levelcontrol frameworks, and new methods to detect and disrupt conversationaldeception powered by generative AI.