Adjusting Image Attributes of Localized Regions with Low-level Dialogue

  • 2020-02-11 20:59:34
  • Tzu-Hsiang Lin, Alexander Rudnicky, Trung Bui, Doo Soon Kim, Jean Oh
  • 3

Abstract

Natural Language Image Editing (NLIE) aims to use natural languageinstructions to edit images. Since novices are inexperienced with image editingtechniques, their instructions are often ambiguous and contain high-levelabstractions that tend to correspond to complex editing steps to accomplish.Motivated by this inexperience aspect, we aim to smooth the learning curve byteaching the novices to edit images using low-level commanding terminologies.Towards this end, we develop a task-oriented dialogue system to investigatelow-level instructions for NLIE. Our system grounds language on the level ofedit operations, and suggests options for a user to choose from. Thoughcompelled to express in low-level terms, a user evaluation shows that 25% ofusers found our system easy-to-use, resonating with our motivation. An analysisshows that users generally adapt to utilizing the proposed low-level languageinterface. In this study, we identify that object segmentation as the keyfactor to the user satisfaction. Our work demonstrates the advantages of thelow-level, direct language-action mapping approach that can be applied to otherproblem domains beyond image editing such as audio editing or industrialdesign.

 

Quick Read (beta)

loading the full paper ...