Repetition Probability as a Self-Reinforcing Feedback Loop

P(repeat | context) rises monotonically after line 3,501 · Apple ML "Learning to Break the Loop" (2024)

Key finding: Once the repetition probability crosses ~0.15, the model enters a regime where each repetition increases the probability of the next repetition. This positive feedback loop is self-reinforcing and difficult to break without external intervention.
Session data: P(Done | context) reached ~0.85 by line 4,200. P(Let me check | context) ≈ 0.72 by line 5,000.

Apple ML Research (2024) "Learning to Break the Loop: Repetition Control in Autoregressive Models"
Holtzman et al. (2019) "The Curious Case of Neural Text Degeneration" ICLR 2020 (nucleus sampling)