Abstract
In this work, we provide a systematic survey of Discrete Diffusion LanguageModels (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs).Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token,parallel decoding paradigm using full attention and a denoising-basedgeneration strategy. This paradigm naturally enables parallel generation,fine-grained output controllability, and dynamic, response-aware perception.These capabilities are previously difficult to achieve with AR models.Recently, a growing number of industrial-scale proprietary d(M)LLMs, as well asa large number of open-source academic d(M)LLMs, have demonstrated performancecomparable to their autoregressive counterparts, while achieving up to 10xacceleration in inference speed. The advancement of discrete diffusion LLMs and MLLMs has been largely drivenby progress in two domains. The first is the development of autoregressive LLMsand MLLMs, which has accumulated vast amounts of data, benchmarks, andfoundational infrastructure for training and inference. The second contributingdomain is the evolution of the mathematical models underlying discretediffusion. Together, these advancements have catalyzed a surge in dLLMs anddMLLMs research in early 2025. In this work, we present a comprehensive overview of the research in the dLLMand dMLLM domains. We trace the historical development of dLLMs and dMLLMs,formalize the underlying mathematical frameworks, and categorize representativemodels. We further analyze key techniques for training and inference, andsummarize emerging applications across language, vision-language, andbiological domains. We conclude by discussing future directions for researchand deployment. Paper collection: https://github.com/LiQiiiii/DLLM-Survey