Decoding-aware sequence-level losses are proved unbounded below: greedy and sampling objectives both diverge, so teacher forcing stays the practical choice for supervised autoregressive training
Autoregressive language models are usually trained by teacher forcing, predicting each token from the ground-truth prefix, while at inference they condition on their own generated prefixes, a mismatch called exposure bias. Two sequence-level losses built to remove that mismatch, one based on greedy decoding and one on autoregressive sampling, are both shown to be unbounded below and therefore unusable for learning. An autoregressive Gumbel trick shows the sampling loss is the expected greedy loss under Gumbel perturbations; a stop-gradient moment-matching direction is not the gradient of any loss yet is a descent direction for the autoregressive negative log-likelihood in function space, at the cost of high variance.