A small team of AI researchers from Carnegie Mellon University, Stanford University, Harvard University and Princeton University, all in the U.S., has found that if large language models are over-trained, it might make them harder to fine-tune. In their paper posted on the arXiv preprint server, the group compared the impact of different amounts of training on a single LLM.
We’ve seen similar effects in the context of reinforcement learning (see the “primacy bias” works of Evgenii Nikishin). It makes sense that it would also apply to LLMs, and any other ML model.