Abstract
Social biases can manifest in language agency. For instance, Whiteindividuals and men are often described as "agentic" and achievement-oriented,whereas Black individuals and women are frequently described as "communal" andas assisting roles. This study establishes agency as an important aspect ofstudying social biases in both human-written and Large Language Model(LLM)-generated texts. To accurately measure "language agency" at sentencelevel, we propose a Language Agency Classification dataset to train reliableagency classifiers. We then use an agency classifier to reveal notable languageagency biases in 6 datasets of human- or LLM-written texts, includingbiographies, professor reviews, and reference letters. While most prior NLPresearch on agency biases focused on single dimensions, we comprehensivelyexplore language agency biases in gender, race, and intersectional identities.We observe that (1) language agency biases in human-written texts align withreal-world social observations; (2) LLM-generated texts demonstrate remarkablyhigher levels of language agency bias than human-written texts; and (3)critical biases in language agency target people of minority groups--forinstance, languages used to describe Black females exhibit the lowest level ofagency across datasets. Our findings reveal intricate social biases in human-and LLM-written texts through the lens of language agency, warning againstusing LLM generations in social contexts without scrutiny.