Need help with GazeCorrection?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

zhangqianhui
131 Stars 25 Forks Apache License 2.0 115 Commits 5 Opened issues

Description

GazeCorrection: Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks

Services available

!
?

Need anything else?

Contributors list

# 2,657
Tensorf...
c-plus-...
C
Android
81 commits
# 573,456
Redis
php7
PHP
Shell
3 commits

GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks

V2 version can be found in GazeAnimation

Official code of paper GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks.


Paper

Abstract

Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem. Gaze correction has a wide range of applications in real life, such as taking a picture with staring at the camera. In this paper, we propose a novel method that is based on the inpainting model to learn from the face image to fill in the missing eye regions with new contents representing corrected eye gaze. Moreover, our model does not require the training dataset labeled with the specific head pose and eye angle information, thus, the training data is easy to collect. To retain the identity information of the eye region in the original input, we propose a self-guided pretrained model to learn the angle-invariance feature. Experiments show our model achieves very compelling gaze-corrected results in the wild dataset which is collected from the website and will be introduced in details.

Citation

If you find this work useful for your research, please cite our paper:

@article{zhang2019gazecorrection,
  title={GazeCorrection: Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks},
  author={Zhang, Jichao and Sun, Meng and Chen, Jingjing and Tang, Hao and Yan, Yan and Qin, Xueying and Sebe, Nicu},
  journal={arXiv preprint arXiv:1906.00805},
  year={2019}
}

Network Architecture

Dependencies

pip install -r requirements.txt

Usage

  • Clone this repo:
    bash
    git clone https://github.com/zhangqianhui/GazeCorrection.git
    
  • Download the NewGaze dataset

Download the tar of NewGaze dataset from Google Driver Linking.

  cd your_path
  unzip NewGazeData.tar

Please edit the options.py to change your dataset path

  • Pretrained Self-Guided Model

We have provided the self-guided pretraining model in directory: ./sgpremodel_g

  • Train this model

(1)Please edit the config.py file to select the proper hyper-parameters.

(2)Change the "basepath" to "yourpath" of NewGaze dataset.

Then

  python train.py --use_sp --is_ss --gpu_id='0' --exper_name='log3_25_1' --batch_size=8 --test_sample_dir='test_sample_dir'

or

  bash scripts/train_log20_3_25_1.sh
  • Test this model

You can download the pretrained model from Google Driver Linking.

  python test.py --gpu_id='0' --exper_name='log3_25_1' --batch_size=8 --test_sample_dir='test_sample_dir'

or

  bash scripts/test_log20_3_25_1.sh

Experiments

  • Comparison Results

  • Experiments Results

More results(GiF)

Reference code

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.