Many machine learning tasks -- particularly those in affective computing --are inherently subjective. When asked to classify facial expressions or to ratean individual's attractiveness, humans may disagree with one another, and nosingle answer may be objectively correct. However, machine learning datasetscommonly have just one "ground truth" label for each sample, so models trainedon these labels may not perform well on tasks that are subjective in nature.Though allowing models to learn from the individual annotators' ratings mayhelp, most datasets do not provide annotator-specific labels for each sample.To address this issue, we propose SeedBERT, a method for recovering annotatorrating distributions from a single label by inducing pre-trained models toattend to different portions of the input. Our human evaluations indicate thatSeedBERT's attention mechanism is consistent with human sources of annotatordisagreement. Moreover, in our empirical evaluations using large languagemodels, SeedBERT demonstrates substantial gains in performance on downstreamsubjective tasks compared both to standard deep learning models and to othercurrent models that account explicitly for annotator disagreement.