Abstract
In this work, we provide a systematic survey of Discrete Diffusion LanguageModels (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs).Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token,parallel decoding paradigm using full attention and a denoising-basedgeneration strategy. This paradigm naturally enables parallel generation,fine-grained output control, and dynamic perception. These capabilities arepreviously difficult to achieve with AR models. A growing number ofindustrial-scale proprietary d(M)LLMs, as well as a large number of open-sourceacademic d(M)LLMs, have demonstrated performance comparable to theirautoregressive counterparts, while achieving up to 10$\times$ acceleration ininference speed. These developments position discrete diffusion models as apromising alternative to intelligence based on the traditional autoregressiveapproach. In this work, we present a comprehensive overview of the research inthe dLLM and dMLLM domains. We trace the historical development of dLLMs anddMLLMs, formalize the underlying mathematical frameworks, list commonly-usedmodeling methods, and categorize representative models. We further analyze keytechniques for training, inference, quantization. We also discuss thetrustworthy issues and summarize emerging applications across language,vision-language, and biological domains and etc.. We conclude by discussingfuture directions for research and deployment. Relative papers are collected inhttps://github.com/LiQiiiii/Awesome-Discrete-Diffusion-LLM_MLLM