Abstract
This article explores the zero-shot performance of state-of-the-art largelanguage models (LLMs) on one of the most challenging tasks in authorshipanalysis: sentence-level style change detection. Benchmarking four LLMs on theofficial PAN~2024 and 2025 "Multi-Author Writing Style Analysis" datasets, wepresent several observations. First, state-of-the-art generative models aresensitive to variations in writing style - even at the granular level ofindividual sentences. Second, their accuracy establishes a challenging baselinefor the task, outperforming suggested baselines of the PAN competition.Finally, we explore the influence of semantics on model predictions and presentevidence suggesting that the latest generation of LLMs may be more sensitive tocontent-independent and purely stylistic signals than previously reported.