LLaMA Pro

The article introduces LLaMA Pro, a novel post-pretraining method for Large Language Models (LLMs) that expands Transformer blocks. Unlike other LLMs, which can forget old skills when acquiring new ones,…