Abstract
We present a novel approach to neural code generation that incorporatesreal-time execution signals into the language model generation process. Whilelarge language models (LLMs) have demonstrated impressive code generationcapabilities, they typically do not utilize execution feedback duringinference, a critical signal that human programmers regularly leverage. Ourmethod, Execution-Guided Classifier-Free Guidance (EG-CFG), dynamicallyincorporates execution signals as the model generates code, providingline-by-line feedback that guides the generation process toward executablesolutions. EG-CFG employs a multi-stage process: first, we conduct beam searchto sample candidate program completions for each line; second, we extractexecution signals by executing these candidates against test cases; andfinally, we incorporate these signals into the prompt during generation. Bymaintaining consistent signals across tokens within the same line andrefreshing signals at line boundaries, our approach provides coherent guidancewhile preserving syntactic structure. Moreover, the method naturally supportsnative parallelism at the task level in which multiple agents operate inparallel, exploring diverse reasoning paths and collectively generating a broadset of candidate solutions. Our experiments across diverse coding tasksdemonstrate that EG-CFG significantly improves code generation performancecompared to standard approaches, achieving state-of-the-art results acrossvarious levels of complexity, from foundational problems to challengingcompetitive programming tasks. Our code is available at:https://github.com/boazlavon/eg_cfg