The paper discusses the need for a localized Large Language Model (LLM) for Arabic, a language with unique cultural characteristics not adequately addressed by current models like ChatGPT. The study introduces a solution called AceGPT, which uses pre-training with Arabic texts, supervised fine-tuning using native Arabic instructions, and reinforcement learning with AI feedback. The goal is to train culturally aware and value-aligned Arabic LLMs for diverse application-specific needs of Arabic-speaking communities. The resulting LLM, AceGPT, has shown promising results in various benchmarks, outperforming ChatGPT in the Vicuna-80 benchmark when evaluated with GPT-4.
Publication date: 21 Sep 2023
Project Page: https://github.com/FreedomIntelligence/AceGPT
Paper: https://arxiv.org/pdf/2309.12053
