Abstract
Newberry et al. (Detecting evolutionary forces in language change, Nature551, 2017) tackle an important but difficult problem in linguistics, thetesting of selective theories of language change against a null model of drift.Having applied a test from population genetics (the Frequency Increment Test)to a number of relevant examples, they suggest stochasticity has a previouslyunder-appreciated role in language evolution. We replicate their results andfind that while the overall observation holds, results produced by thisapproach on individual time series are highly sensitive to how the corpus isorganized into temporal segments (binning). Furthermore, we use a large set ofsimulations in conjunction with binning to systematically explore the range ofapplicability of the FIT. The approach proposed by Newberry et al. provides asystematic way of generating hypotheses about language change, marking anotherstep forward in big-data driven linguistic research. However, along with thepossibilities, the limitations of the approach need to be appreciated. Cautionshould be exercised with interpreting results of the FIT (and similar tests) onindividual series, given the demonstrable limitations, and fundamentaldifferences between genetic and linguistic data. Our findings also haveimplications for selection testing and temporal binning in general.