The paper presents AlignProp, a method for aligning text-to-image diffusion models to desired reward functions via backpropagation. These models are useful in image generation but controlling their behavior for specific tasks can be challenging. The proposed method overcomes this by adapting the models to various objectives such as image-text alignment, aesthetics, and controllability of the number of objects present. The authors claim that AlignProp achieves higher rewards in fewer training steps than alternatives and is conceptually simpler, making it a preferred choice for optimizing diffusion models.
Publication date: 5 Oct 2023
Project Page: https://align-prop.github.io/
Paper: https://arxiv.org/pdf/2310.03739
