Abstract
Styled Handwritten Text Generation (HTG) has recently received attention fromthe computer vision and document analysis communities, which have developedseveral solutions, either GAN- or diffusion-based, that achieved promisingresults. Nonetheless, these strategies fail to generalize to novel styles andhave technical constraints, particularly in terms of maximum output length andtraining efficiency. To overcome these limitations, in this work, we propose anovel framework for text image generation, dubbed Emuru. Our approach leveragesa powerful text image representation model (a variational autoencoder) combinedwith an autoregressive Transformer. Our approach enables the generation ofstyled text images conditioned on textual content and style examples, such asspecific fonts or handwriting styles. We train our model solely on a diverse,synthetic dataset of English text rendered in over 100,000 typewritten andcalligraphy fonts, which gives it the capability to reproduce unseen styles(both fonts and users' handwriting) in zero-shot. To the best of our knowledge,Emuru is the first autoregressive model for HTG, and the first designedspecifically for generalization to novel styles. Moreover, our model generatesimages without background artifacts, which are easier to use for downstreamapplications. Extensive evaluation on both typewritten and handwritten,any-length text image generation scenarios demonstrates the effectiveness ofour approach.