Human communication is increasingly intermixed with language generated by AI.Across chat, email, and social media, AI systems suggest words, completesentences, or produce entire conversations. AI-generated language is often notidentified as such but presented as language written by humans, raisingconcerns about novel forms of deception and manipulation. Here, we study howhumans discern whether verbal self-presentations, one of the most personal andconsequential forms of language, were generated by AI. In six experiments,participants (N = 4,600) were unable to detect self-presentations generated bystate-of-the-art AI language models in professional, hospitality, and datingcontexts. A computational analysis of language features shows that humanjudgments of AI-generated language are hindered by intuitive but flawedheuristics such as associating first-person pronouns, use of contractions, orfamily topics with human-written language. We experimentally demonstrate thatthese heuristics make human judgment of AI-generated language predictable andmanipulable, allowing AI systems to produce text perceived as "more human thanhuman." We discuss solutions, such as AI accents, to reduce the deceptivepotential of language generated by AI, limiting the subversion of humanintuition.