Abstract
In-Context Learning has shown great potential for aligning Large LanguageModels (LLMs) with human values, helping reduce harmful outputs and accommodatediverse preferences without costly post-training, known as In-Context Alignment(ICA). However, LLMs' comprehension of input prompts remains agnostic, limitingICA's ability to address value tensions--human values are inherentlypluralistic, often imposing conflicting demands, e.g., stimulation vs.tradition. Current ICA methods therefore face the Instruction Bottleneckchallenge, where LLMs struggle to reconcile multiple intended values within asingle prompt, leading to incomplete or biased alignment. To address this, wepropose PICACO, a novel pluralistic ICA method. Without fine-tuning, PICACOoptimizes a meta-instruction that navigates multiple values to better elicitLLMs' understanding of them and improve their alignment. This is achieved bymaximizing the total correlation between specified values and LLM responses,theoretically reinforcing value correlation while reducing distractive noise,resulting in effective value instructions. Extensive experiments on five valuesets show that PICACO works well with both black-box and open-source LLMs,outperforms several recent strong baselines, and achieves a better balanceacross up to 8 distinct values.