Abstract
Deep learning has recently made remarkable progress in natural languageprocessing. Yet, the resulting algorithms remain far from competing with thelanguage abilities of the human brain. Predictive coding theory offers apotential explanation to this discrepancy: while deep language algorithms areoptimized to predict adjacent words, the human brain would be tuned to makelong-range and hierarchical predictions. To test this hypothesis, we analyzethe fMRI brain signals of 304 subjects each listening to 70min of shortstories. After confirming that the activations of deep language algorithmslinearly map onto those of the brain, we show that enhancing these models withlong-range forecast representations improves their brain-mapping. The resultsfurther reveal a hierarchy of predictions in the brain, whereby thefronto-parietal cortices forecast more abstract and more distantrepresentations than the temporal cortices. Overall, this study strengthenspredictive coding theory and suggests a critical role of long-range andhierarchical predictions in natural language processing.