Perturbed Gradient Descent

The paper presents PAC-tuning, a method for fine-tuning pretrained language models (PLMs) to enhance their generalization performance. The method uses PAC-Bayes training to directly minimize the PAC-Bayes generalization bound, resulting…