Abstract
Steering vectors are a lightweight method for controlling text properties byadding a learned bias to language model activations at inference time. So far,steering vectors have predominantly been evaluated in multiple-choice settings,while their effectiveness in free-form generation tasks remains understudied.Moving "Beyond Multiple Choice," we thoroughly evaluate the effectiveness ofsteering vectors in adaptively controlling topical focus, sentiment, toxicity,and readability in abstractive summaries of the NEWTS dataset. We find thatsteering effectively controls the targeted summary properties, but highsteering strengths consistently degrade both intrinsic and extrinsic textquality. Compared to steering, prompting offers weaker control, whilepreserving text quality. Combining steering and prompting yields the strongestcontrol over text properties and offers the most favorable efficacy-qualitytrade-off at moderate steering strengths. Our results underscore the practicaltrade-off between control strength and text quality preservation when applyingsteering vectors to free-form generation tasks.