rl-diffusion.github.io - Training Diffusion Models with Reinforcement Learning

home > domains > rl-diffusion.github.io

Example domain paragraphs

UPDATE: We now have a PyTorch implementation that supports LoRA for low-memory training here !

We train diffusion models directly on downstream objectives using reinforcement learning (RL). We do this by posing denoising diffusion as a multi-step decision-making problem, enabling a class of policy gradient algorithms that we call denoising diffusion policy optimization (DDPO). We use DDPO to finetune Stable Diffusion on objectives that are difficult to express via prompting, such as image compressibility, and those derived from human feedback, such as aesthetic quality. We also show that DDPO can be

We use feedback from a large vision-language model, LLaVA , to automatically improve prompt-image alignment.

Links to rl-diffusion.github.io (8)

• yilundu.github.io Yilun Du
• kevin.black Kevin Black
• jannerm.github.io Michael Janner
• residual-assembly.github.io From Imitation to Refinement—Residual RL for Precise Visual Assembly
• kevinblack.dev Kevin Black
• readout-guidance.github.io Readout Guidance: Learning Control from Diffusion Features
• mjanner.com Michael Janner
• devinluo27.github.io Yunhao Luo @ Brown