Abstract
We propose eXtensible Prompt (X-Prompt) for prompting a large language model(LLM) beyond natural language (NL). X-Prompt instructs an LLM with not only NLbut also an extensible vocabulary of imaginary words. Registering new imaginarywords allows us to instruct the LLM to comprehend concepts that are difficultto describe with NL words, thereby making a prompt more descriptive. Also,these imaginary words are designed to be out-of-distribution (OOD) robust sothat they can be (re)used like NL words in various prompts, distinguishingX-Prompt from soft prompt that is for fitting in-distribution data. We proposecontext-augmented learning (CAL) to learn imaginary words for generalusability, enabling them to work properly in OOD (unseen) prompts. Weexperiment X-Prompt for zero-shot language style customization as a case study.The promising results of X-Prompt demonstrate its potential to facilitateadvanced interaction beyond the natural language interface, bridging thecommunication gap between humans and LLMs.