Abstract
Multimodal Large Language Models (MLLMs) have demonstrated remarkablecapabilities in automated front-end engineering, e.g., generating UI code fromvisual designs. However, existing front-end UI code generation benchmarks havethe following limitations: (1) While framework-based development becomespredominant in modern front-end programming, current benchmarks fail toincorporate mainstream development frameworks. (2) Existing evaluations focussolely on the UI code generation task, whereas practical UI developmentinvolves several iterations, including refining editing, and repairing issues.(3) Current benchmarks employ unidimensional evaluation, lacking investigationinto influencing factors like task difficulty, input context variations, andin-depth code-level analysis. To bridge these gaps, we introduce DesignBench, amulti-framework, multi-task evaluation benchmark for assessing MLLMs'capabilities in automated front-end engineering. DesignBench encompasses threewidely-used UI frameworks (React, Vue, and Angular) alongside vanilla HTML/CSS,and evaluates on three essential front-end tasks (generation, edit, and repair)in real-world development workflows. DesignBench contains 900 webpage samplesspanning over 11 topics, 9 edit types, and 6 issue categories, enablingdetailed analysis of MLLM performance across multiple dimensions. Oursystematic evaluation reveals critical insights into MLLMs' framework-specificlimitations, task-related bottlenecks, and performance variations underdifferent conditions, providing guidance for future research in automatedfront-end development. Our code and data are available athttps://github.com/WebPAI/DesignBench.