Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
The study introduces a new approach, Parameter-Efficient Sparsity Crafting (PESC), to improve the performance of Large Language Models (LLMs) in instruction tuning tasks. Instruction tuning enhances the ability of LLMs…
Continue reading