Need help with rt_gene?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Tobias-Fischer
148 Stars 35 Forks Other 257 Commits 0 Opened issues

Description

RT-GENE: Real-Time Eye Gaze and Blink Estimation in Natural Environments

Services available

!
?

Need anything else?

Contributors list

# 86,354
Shell
MATLAB
control...
pixhawk
157 commits
# 718,992
Python
MATLAB
Jupyter...
Shell
1 commit
# 207,284
Rust
travis-...
webfram...
webrtc
1 commit

RT-GENE & RT-BENE: Real-Time Eye Gaze and Blink Estimation in Natural Environments

License: CC BY-NC-SA 4.0 HitCount stars GitHub issues GitHub repo size

PWC PWC PWC

PWC PWC PWC

This repository contains code and dataset references for two papers: RT-GENE (Gaze Estimation; ECCV2018) and RT-BENE (Blink Estimation; ICCV2019 Workshops).

RT-GENE (Gaze Estimation)

License + Attribution

The RT-GENE code is licensed under CC BY-NC-SA 4.0. Commercial usage is not permitted. If you use this dataset or the code in a scientific publication, please cite the following paper:

Paper abstract

@inproceedings{FischerECCV2018,
author = {Tobias Fischer and Hyung Jin Chang and Yiannis Demiris},
title = {{RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments}},
booktitle = {European Conference on Computer Vision},
year = {2018},
month = {September},
pages = {339--357}
}

This work was supported in part by the Samsung Global Research Outreach program, and in part by the EU Horizon 2020 Project PAL (643783-RIA).

Overview + Accompanying Dataset

The code is split into four parts, each having its own README contained. There is also an accompanying dataset (alternative link) to the code. For more information, other datasets and more open-source software please visit the Personal Robotic Lab's website: https://www.imperial.ac.uk/personal-robotics/software/.

RT-GENE ROS package

The rt_gene directory contains a ROS package for real-time eye gaze and blink estimation. This contains all the code required at inference time.

RT-GENE inference example

RT-GENE Standalone Version

The rtgenestandalone directory contains instructions for eye gaze estimation given a set of images. It shares code with the

rt_gene
package (above), in particular the code in rtgene/src/rtgene.

RT-GENE Inpainting

The rtgeneinpainting directory contains code to inpaint the region covered by the eyetracking glasses.

Inpaining example

RT-GENE Model Training

The rtgenemodel_training directory allows using the inpainted images to train a deep neural network for eye gaze estimation.

Accuracy on RT-GENE dataset

RT-BENE (Blink Estimation)

License + Attribution

The RT-BENE code is licensed under CC BY-NC-SA 4.0. Commercial usage is not permitted. If you use our blink estimation code or dataset, please cite the relevant paper:

@inproceedings{CortaceroICCV2019W,
author={Kevin Cortacero and Tobias Fischer and Yiannis Demiris},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision Workshops},
title = {RT-BENE: A Dataset and Baselines for Real-Time Blink Estimation in Natural Environments},
year = {2019},
}

RT-BENE was supported by the EU Horizon 2020 Project PAL (643783-RIA) and a Royal Academy of Engineering Chair in Emerging Technologies to Yiannis Demiris.

Overview + Accompanying Dataset

The code is split into several parts, each having its own README. There is also an associated RT-BENE dataset. For more information, other datasets and more open-source software please visit the Personal Robotic Lab's website: https://www.imperial.ac.uk/personal-robotics/software/. Please note that a lot of the code is shared with RT-GENE (see above), hence there are many references to RT-GENE below.

Paper overview

RT-BENE ROS package

The rt_gene directory contains a ROS package for real-time eye gaze and blink estimation. This contains all the code required at inference time. For blink estimation, please refer to the estimate_blink.py file.

RT-BENE inference example

RT-BENE Standalone Version

The rtbenestandalone directory contains instructions for blink estimation given a set of images. It makes use of the code in rtgene/src/rtbene.

RT-BENE Model Training

The rtbenemodel_training directory contains the code required to train models with the labels contained in the RT-BENE dataset (see below). We will soon at evaluation code in this directory, too.

RT-BENE Dataset

RT-BENE labels

We manually annotated images contained in the "noglasses" part of the RT-GENE dataset. The RT-BENE dataset on Zenodo contains the eye image patches and associated annotations to train the blink models.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.