Abstract
Auto-bidding is an essential tool for advertisers to enhance theiradvertising performance. Recent progress has shown that AI-Generated Bidding(AIGB), which formulates the auto-bidding as a trajectory generation task andtrains a conditional diffusion-based planner on offline data, achieves superiorand stable performance compared to typical offline reinforcement learning(RL)-based auto-bidding methods. However, existing AIGB methods still encountera performance bottleneck due to their neglect of fine-grained generationquality evaluation and inability to explore beyond static datasets. To addressthis, we propose AIGB-Pearl (\emph{Planning with EvAluator via RL}), a novelmethod that integrates generative planning and policy optimization. The key toAIGB-Pearl is to construct a non-bootstrapped \emph{trajectory evaluator} toassign rewards and guide policy search, enabling the planner to optimize itsgeneration quality iteratively through interaction. Furthermore, to enhancetrajectory evaluator accuracy in offline settings, we incorporate three keytechniques: (i) a Large Language Model (LLM)-based architecture for betterrepresentational capacity, (ii) hybrid point-wise and pair-wise losses forbetter score learning, and (iii) adaptive integration of expert feedback forbetter generalization ability. Extensive experiments on both simulated andreal-world advertising systems demonstrate the state-of-the-art performance ofour approach.