energy-based models

Why next-token models can plan ahead

This paper argues autoregressive language models can exhibit lookahead behavior despite training only on next-token prediction.