In-Domain GAN Inversion for Real Image Editing

Figure: Real image editing using the proposed In-Domain GAN inversion with a fixed GAN generator.

In-Domain GAN Inversion for Real Image Editing
Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou
European Conference on Computer Vision (ECCV) 2020

[Paper] [Project Page] [Demo] [Colab]

NOTE: This repository is a simple PyTorch version of this repo, and ONLY supports inference.

Editing Tasks

Pre-trained Models

Please download the pre-trained models from the following links and save them to

models/pretrain/

| Description | Generator | Encoder | | :---------- | :-------- | :------ | | Model trained on FFHQ dataset. | face256x256generator | face256x256encoder | Model trained on LSUN Tower dataset. | tower256x256generator | tower256x256encoder | Model trained on LSUN Bedroom dataset. | bedroom256x256generator | bedroom256x256encoder | Perceptual Model

Inversion

MODEL_NAME='styleganinv_ffhq256'
IMAGE_LIST='examples/test.list'
python invert.py $MODEL_NAME $IMAGE_LIST

NOTE: We find that 100 iterations are good enough for inverting an image, which takes about 8s (on P40). But users can always use more iterations (much slower) for a more precise reconstruction.

Semantic Diffusion

MODEL_NAME='styleganinv_ffhq256'
TARGET_LIST='examples/target.list'
CONTEXT_LIST='examples/context.list'
python diffuse.py $MODEL_NAME $TARGET_LIST $CONTEXT_LIST

NOTE: The diffusion process is highly similar to image inversion. The main difference is that only the target patch is used to compute loss for masked optimization.

Interpolation

SRC_DIR='results/inversion/test'
DST_DIR='results/inversion/test'
python interpolate.py $MODEL_NAME $SRC_DIR $DST_DIR

Manipulation

IMAGE_DIR='results/inversion/test'
BOUNDARY='boundaries/expression.npy'
python manipulate.py $MODEL_NAME $IMAGE_DIR $BOUNDARY

NOTE: Boundaries are obtained using InterFaceGAN.

Style Mixing

STYLE_DIR='results/inversion/test'
CONTENT_DIR='results/inversion/test'
python mix_style.py $MODEL_NAME $STYLE_DIR $CONTENT_DIR

BibTeX

@inproceedings{zhu2020indomain,
  title     = {In-domain GAN Inversion for Real Image Editing},
  author    = {Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year      = {2020}
}