Researchers warn of ‘catastrophic overtraining’ in LLMs

A brand new tutorial research challenges a core assumption in growing massive language fashions (LLMs), warning that extra pre-training knowledge could not all the time result in higher fashions.

Researchers from a few of the main pc science establishments within the West and around the globe—together with Carnegie Mellon College, Stanford College, Harvard College and Princeton College—have launched the idea of “Catastrophic Overtraining. ” They present that prolonged pre-training can really make language fashions more durable to fine-tune, in the end degrading their efficiency.

The research, “Overtrained Language Fashions Are More durable to Advantageous-Tune,” is on the market on arXiv and led by Jacob Mitchell Springer. Its co-authors are Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig and Aditi Raghunathan.

The regulation of diminishing returns

The analysis focuses on a shocking pattern noticed in trendy LLM growth: whereas fashions are pre-trained on ever-expanding swimming pools of knowledge—licensed or scraped from the net, represented to an LLM as a sequence of tokens or numerical representations of ideas and concepts—rising the token quantity throughout pre-training could result in lowered effectiveness when these fashions are later fine-tuned for particular duties.

The group performed a sequence of empirical evaluations and theoretical analyses to look at the impact of prolonged pre-training on mannequin adaptability.

One of many key findings facilities on AI2’s open supply OLMo-1B mannequin.

The researchers in contrast two variations of this mannequin: one pre-trained on 2.3 trillion tokens and one other on 3 trillion tokens.

Regardless of the latter being skilled on 30% extra knowledge, the latter mannequin carried out worse after instruction tuning. Particularly, the 3T-token mannequin confirmed over 2% worse efficiency on a number of commonplace language mannequin benchmarks in comparison with its 2.3T-token counterpart. In some evaluations, the degradation in efficiency reached as much as 3%.

The researchers argue that this decline will not be an anomaly however relatively a constant phenomenon they time period “Catastrophic Overtraining.”

Understanding sensitivity and forgetting

The paper attributes this degradation to a scientific enhance in what they name “progressive sensitivity.” As fashions endure prolonged pre-training, their parameters change into extra delicate to modifications.

This elevated fragility makes them extra weak to degradation throughout post-training modifications reminiscent of instruction tuning, fine-tuning for multimodal duties, and even easy weight perturbations.

The researchers present proof that, past a sure level in pre-training, any modification—whether or not structured like fine-tuning or unstructured like including Gaussian noise—results in a larger lack of beforehand realized capabilities.

This sensitivity ends in “forgetting,” the place the mannequin’s authentic strengths deteriorate as new coaching knowledge is launched.

The research identifies an “inflection level” in pre-training, after which extra coaching results in diminishing and even damaging returns relating to fine-tuning outcomes. For the OLMo-1B mannequin, this threshold emerged round 2.5 trillion tokens.

A wealth of proof

The group’s evaluation spans real-world and managed experimental settings. They examined the phenomenon throughout totally different duties, together with instruction tuning utilizing datasets like Anthropic-HH and TULU and multimodal fine-tuning utilizing the LLaVA framework.

The outcomes persistently confirmed that fashions pre-trained past sure token budgets underperformed after fine-tuning.

Moreover, the researchers constructed a theoretical mannequin utilizing linear networks to grasp higher why overtraining results in elevated sensitivity.

Their evaluation confirmed that progressive sensitivity and catastrophic overtraining are mathematically inevitable when pre-training continues indefinitely with out correct constraints.

The last word takeaway? Mannequin suppliers and trainers should make trade-offs

The findings problem the widespread assumption that extra pre-training knowledge is all the time higher. As a substitute, the paper suggests a nuanced trade-off: whereas longer pre-training improves the bottom mannequin’s capabilities, it additionally will increase the danger that fine-tuning will degrade these capabilities.

In follow, makes an attempt to mitigate this impact—reminiscent of adjusting fine-tuning studying charges or including regularization—could delay the onset of catastrophic overtraining however can not absolutely get rid of it with out sacrificing downstream efficiency.

Thus, for enterprises trying to leverage LLMs to enhance enterprise workflows and outcomes, if one thought for doing so is to fine-tune an open-source mannequin, the lesson from this analysis signifies that fine-tuning decrease parameter fashions skilled on much less materials is more likely to arrive at a extra dependable manufacturing mannequin.

The authors acknowledge that additional analysis is required to grasp the components influencing when and the way catastrophic overtraining happens. Open questions embody whether or not the pre-training optimizer, coaching goal, or knowledge distribution can impression the severity of the phenomenon.

Implications for future LLM and AI mannequin growth

The research considerably impacts how organizations and researchers design and prepare massive language fashions. As the sector continues to pursue bigger and extra succesful fashions, this analysis highlights the significance of balancing pre-training period with post-training adaptability.

Moreover, the findings could affect how mannequin builders take into consideration useful resource allocation. Fairly than focusing completely on rising pre-training budgets, builders could must reassess methods to optimize downstream efficiency with out incurring the damaging results of catastrophic overtraining.