Abstract

This Proximal Policy Optimization (PPO) is widely used for autonomous driving but often faces sparse rewards and unsafe exploration, which are critical issues in safety-sensitive scenarios. We propose LLM-PPO Driver, a framework that improves PPO by using a Large Language Model (LLM) as a source of prior driving knowledge during training. The LLM supports learning through reward shaping and imitation learning, without participating in real-time control. Experiments in the Gym highway-v0 environment show that both methods outperform baseline PPO, with imitation learning achieving the best results. These findings demonstrate that LLM-guided prior knowledge can improve safety, task success, and learning efficiency in autonomous driving.