The study tackles the issues in Automated Model Evaluation (AutoEval), such as overconfidence and high computational cost. It introduces a novel measure, Meta-Distribution Energy (MDE), to enhance the AutoEval framework’s efficiency and effectiveness. The MDE builds a meta-distribution statistic, provides a smoother representation through energy-based learning, and connects with classification loss. The researchers validate MDE’s validity and superiority over previous methods through extensive experiments. They also demonstrate MDE’s versatility in integrating with large-scale models and adapting to learning scenarios with noisy or imbalanced labels.
Publication date: 24 Jan 2024
Project Page: https://github.com/pengr/Energy_AutoEval
Paper: https://arxiv.org/pdf/2401.12689
