E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models

This research introduces the Emotional chat Model (E-chat), a new dialogue system that comprehends and reacts to emotions carried in speech. Using an emotion embedding extracted by a speech encoder and combined with Large Language Models, E-chat can respond to different emotional contexts. The E-chat200 dataset, specifically designed for emotion-sensitive spoken dialogue, is also introduced. In various evaluation metrics, the E-chat model consistently outperforms baseline Large Language Models, indicating its potential in emotional comprehension and human-machine interaction.

Publication date: 4 Jan 2024
Project Page: Not provided
Paper: https://arxiv.org/pdf/2401.00475