Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game Perspective

This paper studies inference-time alignment in large-scale AI systems through the lens of Stackelberg game theory. The authors propose a reward-shaping framework that models alignment as a strategic interaction between a leader and follower, enabling more robust and controllable decision-making during inference. The approach introduces theoretically grounded reward modifications that improve alignment performance while preserving policy effectiveness. Experimental evaluations demonstrate that the method enhances alignment robustness across reinforcement learning and generative AI settings.