Abstract
Zipf's law predicts a power-law relationship between word rank and frequencyin language communication systems and has been widely reported in a variety ofnatural language processing applications. However, the emergence of naturallanguage is often modeled as a function of bias between speaker and listenerinterests, which lacks a direct way of relating information-theoretic bias toZipfian rank. A function of bias also serves as an unintuitive interpretationof the communicative effort exchanged between a speaker and a listener. Wecounter these shortcomings by proposing a novel integral transform and kernelfor mapping communicative bias functions to corresponding word frequency-rankrepresentations at any arbitrary phase transition point, resulting in a directway to link communicative effort (modeled by speaker/listener bias) to specificvocabulary used (represented by word rank). We demonstrate the practicalutility of our integral transform by showing how a change from bias to rankresults in greater accuracy and performance at an image classification task forassigning word labels to images randomly subsampled from CIFAR10. We model thistask as a reinforcement learning game between a speaker and listener andcompare the relative impact of bias and Zipfian word rank on communicativeperformance (and accuracy) between the two agents.