This is the official repository for the paper Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models.

🎉 Accepted to ICLR 2025 [OpenReview]

Yongjin Yang*, Sihyeon Kim*, Hojung Jung, Sangmin Bae, Sangmook Kim, Se-Young Yun^ Kimin Lee^ <br/> *: Equal Contribution, ^: Corresponding Author

FiFA is a framework for automatically filtering human feedback data to improve text-to-image diffusion models. Our method helps select high-quality training samples that lead to efficient and effective alignment of text-to-image diffusion models.

1. Create a Conda virtual environment
2. Install dependencies:

pip install -r requirements.txt

• Primary dataset: Pick-a-Pic v2
• Secondary dataset: HPSv2
• Filtered datasets are available in the train_data directory

• Manually filtered Pick-a-Pic v2 test set: test_data/qas_test_filtered.json
• Additional benchmarks in test_data/:
  • PartiPrompts
  • HPD v2 benchmark

Use select_data.py to filter your training data:

• Customize parameters:
  • K: number of filtered data
  • alpha: hyperparameter for text quality
  • beta (gamma in our paper): hyperparameter for text diversity
• Specify input/output paths for datasets

Two training scripts are provided:

• bash_train.sh: For SD1.5 models
• bash_train_sdxl.sh: For SDXL models

• Full dataset: Set train_data_file=""
• Filtered subset: Set train_data_file=path/to/filtered/indices
• Image-caption pairs: Include "pair" in filename
• Caption-based filtering: Include "caption" in filename

• --cache_dir: Pick-a-Pic dataset cache location
• --train_file_list: Selected data index files
• --base_output_dir: Checkpoint and log directory
• --use_new_label_0: Enable reward-based relabeling
• --image_to_reward_path: Path to an image-reward mapping file
• --resume_from_checkpoint: Resume training point
• --checkpointing_steps: Checkpoint frequency

Use bash_test.sh or bash_test_sdxl.sh for model evaluation.

• --reward_type: Reward function selection
• --data_list: Test prompt file suffixes
• --pretrained_model_name_or_path: Model path/URL
• --model_list: Fine-tuned model checkpoints
• --output-dir: Results directory
• --checkpoint_list: Model checkpoints to evaluate
• --num_imgs_per_prompt: Images per prompt

If you use this code in your research, please cite our paper:

@inproceedings{
yang2025automated,
title={Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models},
author={Yongjin Yang and Sihyeon Kim and Hojung Jung and Sangmin Bae and SangMook Kim and Se-Young Yun and Kimin Lee},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=8jvVNPHtVJ}
}

Our code is based on Diffusion-DPO. We thank the authors for releasing their code. If you use our model and code, please consider citing these works as well.