Abstract
What can we learn about language from studying how it is used by ChatGPT andother large language model (LLM)-based chatbots? In this paper, we analyse thedistinctive character of language generated by ChatGPT, in relation toquestions raised by natural language processing pioneer, and student ofWittgenstein, Margaret Masterman. Following frequent complaints that LLM-basedchatbots produce "slop," or even "bullshit," in the sense of Frankfurt'spopular monograph On Bullshit, we conduct an empirical study to contrast thelanguage of 1,000 scientific publications with typical text generated byChatGPT. We then explore whether the same language features can be detected intwo well-known contexts of social dysfunction: George Orwell's critique ofpolitical speech, and David Graeber's characterisation of bullshit jobs. Usingsimple hypothesis-testing methods, we demonstrate that a statistical model ofsloppy bullshit can reliably relate the Frankfurtian artificial bullshit ofChatGPT to the political and workplace functions of bullshit as observed innatural human language.