Abstract

Imitation Learning (IL) for autonomous driving still suffers from shortcut learning, where policies rely on spurious correlations rather than causal driving behavior. In Transformer-based IL planners, such shortcut learning can appear as attention collapse, where attention weights become excessively concentrated on a small subset of input channels. This weakens robustness in challenging closed-loop scenarios. We propose CAR Planner, which mitigates shortcut learning by imposing a constrained optimization on ego-state cross-attention. The constraint penalizes only excessive concentration of attention weights, preserving task-driven channel-importance ordering while discouraging over-reliance on a few channels. On the nuPlan benchmark, CAR Planner shows substantially less degra- dation under ego-state channel reduction and achieves stronger closed-loop performance in challenging scenarios than strong baselines, with negligible training overhead and no inference- time cost. These results demonstrate that constrained attention is effective for robust imitation learning in autonomous driving.

Additional Comments