Guided-pix2pix

by vt-vl-lab

vt-vl-lab / Guided-pix2pix

[ICCV 2019] Guided Image-to-Image Translation with Bi-Directional Feature Transformation

152 Stars 15 Forks Last release: Not found Other 5 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Guided Image-to-Image Translation with Bi-Directional Feature Transformation

[Project | Paper]

Official Pytorch implementation for Guided Image-to-Image Translation with Bi-Directional Feature Transformation. Please contact Badour AlBahar ([email protected]) if you have any questions.

Prerequisites

This codebase was developed and tested with: - Python2.7 - Pytorch 0.4.1.post2 - CUDA 8.0

Datasets

Train

1. Pose transfer:

python train.py --dataroot /root/DeepFashion/ --name exp_name --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --lr 0.0002 --niter 100 --niter_decay 0 --batch_size 8 --use_GAN --netD basic --beta1 0.9 --checkpoints_dir ./pose_checkpoints

2. Texture transfer:

python train.py --dataroot /root/training_handbags_pretrain/ --name exp_name --netG bFT_unet --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --niter 100 --niter_decay 0 --batch_size 256 --lr 0.0002 --use_GAN --netD basic --n_layers 7 --beta1 .9 --checkpoints_dir ./texture_checkpoints

3. Depth Upsampling:

python train.py --dataroot /root/NYU_RGBD_matfiles/ --name exp_name --netG bFT_resnet --dataset_mode depth --input_nc 1     --guide_nc 3 --output_nc 1 --lr 0.0002 --niter 500 --niter_decay 0 --batch_size 2 --checkpoints_dir ./depth_checkpoints --depthTask_scale [4, 8, or 16]

Test

You can specify which epoch to test by specifying

--epoch
or use the default which is the latest epoch. Results will be saved in
--results_dir
.

1. Pose transfer:

python test.py --dataroot /root/DeepFashion/ --name exp_name --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --checkpoints_dir ./pose_checkpoints --task pose --results_dir ./pose_results

2. Texture transfer:

python test.py --dataroot /root/training_handbags_pretrain/ --name exp_name --netG bFT_unet --n_layers 7 --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --checkpoints_dir ./texture_checkpoints --task texture --results_dir ./texture_results

3. Depth Upsampling:

python test.py --dataroot /root/NYU_RGBD_matfiles/ --name exp_name --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./depth_checkpoints --task depth  --depthTask_scale [4, 8, or 16] --results_dir ./depth_results

Pretrained checkpoints

  • Download the pretrained checkpoints here.

  • Test: For example, to test the depth upsampling task with scale 16:

python test.py --dataroot /root/NYU_RGBD_matfiles/ --name depth_16 --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./checkpoints/pretrained/ --task depth  --depthTask_scale 16 --results_dir ./depth_results

Evaluate

You can specify which epoch to evaluate by specifying

--epoch
or use the default which is the latest epoch. Results will be saved in
--results_dir
.

1. Pose transfer:
Please note that the inception score evaluation requires tensorflow. We evaluate with tensorflow 1.4.0.

python evaluate.py --dataroot /root/DeepFashion/ --name pose --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --checkpoints_dir ./checkpoints/pretrained/ --task pose --results_dir ./pose_results

This will save the results in

--results_dir
and compute both SSIM and IS metrics.

2. Texture transfer: Please download the pretrained model of textureGAN in

./resources
from bags, shoes, and clothes. For example, to test the pretrained texture transfer model for the bags dataset:
python evaluate.py --dataroot /root/training_handbags_pretrain/ --name texture_bags --netG bFT_unet --n_layers 7 --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --checkpoints_dir ./checkpoints/pretrained/ --task texture --results_dir ./texture_results

This will save the output of bFT and textureGAN in

--results_dir
for 10 random input texture patches per test image. The results can then be used to compute FID and LPIPS.

3. Depth Upsampling:

python evaluate.py --dataroot /root/NYU_RGBD_matfiles/ --name depth_16 --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./checkpoints/pretrained/ --task depth  --depthTask_scale 16 --results_dir ./depth_results

This will save the results in

--results_dir
and compute their RMSE metric.

Acknowledgments

This code is heavily borrowed from CycleGAN and pix2pix in PyTorch. We thank Shih-Yang Su for the code review.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.