Abstract
Adapting large language models to multiple tasks can cause cross-skillinterference, where improvements for one skill degrade another. While methodssuch as LoRA impose orthogonality constraints at the weight level, they do notfully address interference in hidden-state representations. We proposeCompositional Subspace Representation Fine-tuning (CS-ReFT), a novelrepresentation-based approach that learns multiple orthonormal subspacetransformations, each specializing in a distinct skill, and composes them via alightweight router. By isolating these subspace edits in the hidden state,rather than weight matrices, CS-ReFT prevents cross-task conflicts moreeffectively. On the AlpacaEval benchmark, applying CS-ReFT to Llama-2-7Bachieves a 93.94% win rate, surpassing GPT-3.5 Turbo (86.30%) while requiringonly 0.0098% of model parameters. These findings show that specializedrepresentation edits, composed via a simple router, significantly enhancemulti-task instruction following with minimal overhead.