An Emulator for Fine-Tuning Large Language Models using Small Language Models
The article presents a new technique called ‘Emulated Fine-Tuning’ (EFT) for large language models. The authors explore the two-stage training pipeline of language models – pre-training and fine-tuning – and…
Continue reading