Abstract
Test-time scaling has emerged as an effective approach for improving languagemodel performance by utilizing additional compute at inference time. Recentstudies have shown that overriding end-of-thinking tokens (e.g., replacing"</think>" with "Wait") can extend reasoning steps and improve accuracy. Inthis work, we explore whether a dedicated continue-thinking token can belearned to trigger extended reasoning. We augment a distilled version ofDeepSeek-R1 with a single learned "<|continue-thinking|>" token, training onlyits embedding via reinforcement learning while keeping the model weightsfrozen. Our experiments show that this learned token achieves improved accuracyon standard math benchmarks compared to both the baseline model and a test-timescaling approach that uses a fixed token (e.g., "Wait") for budget forcing. Inparticular, we observe that in cases where the fixed-token approach enhancesthe base model's accuracy, our method achieves a markedly greater improvement.For example, on the GSM8K benchmark, the fixed-token approach yields a 1.3%absolute improvement in accuracy, whereas our learned-token method achieves a4.2% improvement over the base model that does not use budget forcing.