Low Rank Decomposition

This paper explores the potential to compress Large Language Models (LLMs) using Low Rank Decomposition (LoRD). The researchers found that ranks for linear layers in these models can be reduced…