PSFR-GAN in PyTorch

Progressive Semantic-Aware Style Transformation for Blind Face Restoration
Chaofeng Chen, Xiaoming Li, Lingbo Yang, Xianhui Lin, Lei Zhang, Kwan-Yee K. Wong

Changelog

• 2021.04.26: Add pytorch vgg19 model to GoogleDrive and remove

  --distributed

  option which causes training error.
• 2021.03.22: Update new model at 15 epoch (52.5k iterations).
• 2021.03.19: Add train codes for PSFRGAN and FPN.

Prerequisites and Installation

• Ubuntu 18.04
• CUDA 10.1
  
• Clone this repository

  git clone https://github.com/chaofengc/PSFR-GAN.git
  cd PSFR-GAN
  

• Python 3.7, install required packages by

  pip3 install -r requirements.txt

  
  

Quick Test

Download Pretrain Models and Dataset

Download the pretrained models from the following link and put them to

./pretrain_models

gj2r

Test single image

Run the following script to enhance face(s) in single input

python test_enhance_single_unalign.py --test_img_path ./test_dir/test_hzgg.jpg --results_dir test_hzgg_results --gpus 1

This script do the following things: - Crop and align all the faces from input image, stored at

results_dir/LQ_faces

results_dir/ParseMaps

results_dir/HQ

results_dir/hq_final.jpg

--gpus

<=0

CUDA_VISIBLE_DEVICE

Test image folder

To test multiple images, we first crop out all the faces and align them use the following script.

python align_and_crop_dir.py --src_dir test_dir --results_dir test_dir_align_results

For images (e.g.

multiface_test.jpg

multiface_test_{face_index}.jpg

python test_enhance_dir_align.py --src_dir test_dir_align_results --results_dir test_dir_enhance_results

results_dir/lq

results_dir/parse

results_dir/hq

Additional test script

For your convenience, we also provide script to test multiple unaligned images and paste the enhance results back. Note the paste back operation could be quite slow for large size images containing many faces (dlib takes time to detect faces in large image).

python test_enhance_dir_unalign.py --src_dir test_dir --results_dir test_unalign_results

test_enhance_single_unalign.py

src_dir

Train the Model

Data Preparation

• Download FFHQ and put the images to

  ../datasets/FFHQ/imgs1024

• Download parsing masks (

  512x512

  ) HERE generated by the pretrained FPN and put them to

  ../datasets/FFHQ/masks512

  .

Note: you may change

../datasets/FFHQ

your_own_path/imgs1024

your_own_path/masks512

Train Script for PSFRGAN

Here is an example train script for PSFRGAN:

python train.py --gpus 2 --model enhance --name PSFRGAN_v001 \
    --g_lr 0.0001 --d_lr 0.0004 --beta1 0.5 \
    --gan_mode 'hinge' --lambda_pix 10 --lambda_fm 10 --lambda_ss 1000 \
    --Dinput_nc 22 --D_num 3 --n_layers_D 4 \
    --batch_size 2 --dataset ffhq  --dataroot ../datasets/FFHQ \
    --visual_freq 100 --print_freq 10 #--continue_train

• Please change the

  --name

  option for different experiments. Tensorboard records with the same name will be moved to

  check_points/log_archive

  , and the weight directory will only store weight history of latest experiment with the same name.

• --gpus

  specify number of GPUs used to train. The script will use GPUs with more available memory first. To specify the GPU index, use

  export CUDA_VISIBLE_DEVICES=your_gpu_ids

  before the script.
• Uncomment

  --continue_train

  to resume training. Current codes do not resume the optimizer state.
• It needs at least 8GB memory to train with batch_size=1.

Scripts for FPN

You may also train your own FPN and generate masks for the HQ images by yourself with the following steps:

• Download CelebAHQ-Mask dataset. Generate

  CelebAMask-HQ-mask

  and

  CelebAMask-HQ-mask-color

  with the provided scripts in

  CelebAMask-HQ/face_parsing/Data_preprocessing/

  .
• Train FPN with the following commmand

  python train.py --gpus 1 --model parse --name FPN_v001 \
  --lr 0.0002 --batch_size 8 \
  --dataset celebahqmask --dataroot ../datasets/CelebAMask-HQ \
  --visual_freq 100 --print_freq 10 #--continue_train
  

• Generate parsing masks with your own FPN using the following command:

  python generate_masks.py --save_masks_dir ../datasets/FFHQ/masks512 --batch_size 8 --parse_net_weight path/to/your/own/FPN 
  

Citation

@inproceedings{ChenPSFRGAN,
    author = {Chen, Chaofeng and Li, Xiaoming and Lingbo, Yang and Lin, Xianhui and Zhang, Lei and Wong, KKY},
    title = {Progressive Semantic-Aware Style Transformation for Blind Face Restoration},
    Journal = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2021}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Acknowledgement

This work is inspired by SPADE, and closed related to DFDNet and HiFaceGAN. Our codes largely benefit from CycleGAN.