Abstract
Graphemes of most languages encode pronunciation, though some are moreexplicit than others. Languages like Spanish have a straightforward mappingbetween its graphemes and phonemes, while this mapping is more convoluted forlanguages like English. Spoken languages such as Cantonese present even morechallenges in pronunciation modeling: (1) they do not have a standard writtenform, (2) the closest graphemic origins are logographic Han characters, ofwhich only a subset of these logographic characters implicitly encodespronunciation. In this work, we propose a multimodal approach to predict thepronunciation of Cantonese logographic characters, using neural networks with ageometric representation of logographs and pronunciation of cognates inhistorically related languages. The proposed framework improves performance by18.1% and 25.0% respective to unimodal and multimodal baselines.