Abstract
Generative agents powered by Large Language Models demonstrate human-likecharacteristics through sophisticated natural language interactions. Theirability to assume roles and personalities based on predefined characterbiographies has positioned them as cost-effective substitutes for humanparticipants in social science research. This paper explores the validity ofsuch persona-based agents in representing human populations; we recreate theHEXACO personality inventory experiment by surveying 310 GPT-4 powered agents,conducting factor analysis on their responses, and comparing these results tothe original findings presented by Ashton, Lee, & Goldberg in 2004. Our resultsfound 1) a coherent and reliable personality structure was recoverable from theagents' responses demonstrating partial alignment to the HEXACO framework. 2)the derived personality dimensions were consistent and reliable within GPT-4,when coupled with a sufficiently curated population, and 3) cross-modelanalysis revealed variability in personality profiling, suggestingmodel-specific biases and limitations. We discuss the practical considerationsand challenges encountered during the experiment. This study contributes to theongoing discourse on the potential benefits and limitations of using generativeagents in social science research and provides useful guidance on designingconsistent and representative agent personas to maximise coverage andrepresentation of human personality traits.