Abstract
Artificial intelligence chatbots have achieved unprecedented adoption, withmillions now using these systems for emotional support and companionship incontexts of widespread social isolation and capacity-constrained mental healthservices. While some users report psychological benefits, concerning edge casesare emerging, including reports of suicide, violence, and delusional thinkinglinked to perceived emotional relationships with chatbots. To understand thisnew risk profile we need to consider the interaction between human cognitiveand emotional biases, and chatbot behavioural tendencies such as agreeableness(sycophancy) and adaptability (in-context learning). We argue that individualswith mental health conditions face increased risks of chatbot-induced beliefdestabilization and dependence, owing to altered belief-updating, impairedreality-testing, and social isolation. Current AI safety measures areinadequate to address these interaction-based risks. To address this emergingpublic health concern, we need coordinated action across clinical practice, AIdevelopment, and regulatory frameworks.