The article presents SCEdit, a generative tuning framework for image diffusion models used in tasks like text-to-image generation and controllable image synthesis. This framework focuses on the role of skip connection in U-Net models, revealing that hierarchical features aggregating long-distance information across encoder and decoder greatly impact the content and quality of image generation. SCEdit integrates and edits Skip Connection using a lightweight tuning module named SC-Tuner, which allows for straightforward extension to controllable image synthesis by injecting different conditions. This method reduces training parameters, memory usage, and computational expense, demonstrating superior efficiency and performance.
Publication date: 18 Dec 2023
Project Page: https://scedit.github.io/
Paper: https://arxiv.org/pdf/2312.11392
