Strategic LLM Training: Multi-Token Prediction's Data Efficiency in Mathematical Reasoning

This figure illustrates the profound impact of training scale on multi-token prediction models' performance on GSM8K, highlighting critical data efficiency considerations for mathematical reasoning.