Abstract
The design process of user interfaces (UIs) often begins with articulatinghigh-level design goals. Translating these high-level design goals intoconcrete design mock-ups, however, requires extensive effort and UI designexpertise. To facilitate this process for app designers and developers, weintroduce three deep-learning techniques to create low-fidelity UI mock-upsfrom a natural language phrase that describes the high-level design goal (e.g."pop up displaying an image and other options"). In particular, we contributetwo retrieval-based methods and one generative method, as well aspre-processing and post-processing techniques to ensure the quality of thecreated UI mock-ups. We quantitatively and qualitatively compare and contrasteach method's ability in suggesting coherent, diverse and relevant UI designmock-ups. We further evaluate these methods with 15 professional UI designersand practitioners to understand each method's advantages and disadvantages. Thedesigners responded positively to the potential of these methods for assistingthe design process.