Our new paper sheds light on the process of knowledge acquisition in language models, with practical implications for:
- data curricula
- the challenges of learning new knowledge when fine-tuning
- the emergence of hallucinations.
Nicolas did a great job on the project! See his thread below!
