1: \begin{abstract}

2: Learning-based Text-to-Image (TTI) models like Stable Diffusion have revolutionized the way visual content is generated in various domains. However, recent research has shown that nonnegligible social bias exists in current state-of-the-art TTI systems, which raises important concerns. In this work, we target resolving the social bias in TTI diffusion models. We begin by formalizing the problem setting and 

3: %in light of Safe Latent Diffusion \citep{schramowski2023safe}, we 

4: use the text descriptions of bias groups to establish an unsafe direction for guiding the diffusion process. Next, we simplify the problem into a weight optimization problem and

5: attempt a Reinforcement solver, Policy Gradient, which shows sub-optimal performance with slow convergence. Further, to overcome limitations, we propose an iterative distribution alignment (IDA) method. Despite its simplicity, we show that IDA shows efficiency and fast convergence in resolving the social bias in TTI diffusion models. Our code will be released.

6: 

7: %We begin by formalizing the problem and leveraging Safe Latent Diffusion (Schramowski, 2023) to establish an unsafe direction for guiding the diffusion process using text descriptions of biased groups. Subsequently, we simplify the problem into a weight optimization problem and explore the use of a Reinforcement solver, Policy Gradient, which exhibits sub-optimal performance with slow convergence. To overcome these limitations, we propose an iterative distribution alignment (IDA) method. Through extensive experimentation, we demonstrate that IDA efficiently resolves social bias in TTI diffusion models with fast convergence. The code for our approach will be made publicly available.

8: \end{abstract}

9: