Need help with pix2code?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

11.3K Stars 1.3K Forks Apache License 2.0 26 Commits 1 Opened issues


pix2code: Generating Code from a Graphical User Interface Screenshot

Services available


Need anything else?

Contributors list

# 3,507
11 commits
# 5,967
6 commits
# 10,110
3 commits


Generating Code from a Graphical User Interface Screenshot



Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).


  title={pix2code: Generating Code from a Graphical User Interface Screenshot},
  author={Beltramelli, Tony},
  journal={arXiv preprint arXiv:1705.07962},


The following software is shared for educational purposes only. The author and its affiliated institution are not responsible in any manner whatsoever for any damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of the use or inability to use this software.

The project pix2code is a research project demonstrating an application of deep neural networks to generate code from visual inputs. The current implementation is not, in any way, intended, nor able to generate code in a real-world context. We could not emphasize enough that this project is experimental and shared for educational purposes only. Both the source code and the datasets are provided to foster future research in machine intelligence and are not designed for end users.



  • Python 2 or 3
  • pip

Install dependencies

pip install -r  requirements.txt


Prepare the data: ```sh

reassemble and unzip the data

cd datasets zip -F --out unzip

cd ../model

split training set and evaluation set while ensuring no training example in the evaluation set


./ ../datasets/ios/alldata ./ ../datasets/android/alldata ./ ../datasets/web/alldata

transform images (normalized pixel values and resized pictures) in training dataset to numpy arrays (smaller files if you need to upload the set to train your model in the cloud)


./ ../datasets/ios/trainingset ../datasets/ios/trainingfeatures ./ ../datasets/android/trainingset ../datasets/android/trainingfeatures ./ ../datasets/web/trainingset ../datasets/web/training_features ```

Train the model: ```sh mkdir bin cd model

provide input path to training data and output path to save trained model and metadata


./ ../datasets/web/training_set ../bin

train on images pre-processed as arrays

./ ../datasets/web/training_features ../bin

train with generator to avoid having to fit all the data in memory (RECOMMENDED)

./ ../datasets/web/training_features ../bin 1

train on top of pretrained weights

./ ../datasets/web/training_features ../bin 1 ../bin/pix2code.h5 ```

Generate code for batch of GUIs: ```sh mkdir code cd model

generate DSL code (.gui file), the default search method is greedy


./ ../bin pix2code ../gui_screenshots ../code

equivalent to command above

./ ../bin pix2code ../gui_screenshots ../code greedy

generate DSL code with beam search and a beam width of size 3

./ ../bin pix2code ../gui_screenshots ../code 3 ```

Generate code for a single GUI image: ```sh mkdir code cd model

generate DSL code (.gui file), the default search method is greedy


./ ../bin pix2code ../test_gui.png ../code

equivalent to command above

./ ../bin pix2code ../test_gui.png ../code greedy

generate DSL code with beam search and a beam width of size 3

./ ../bin pix2code ../test_gui.png ../code 3 ```

Compile generated code to target language: ```sh cd compiler

compile .gui file to Android XML UI

./ .gui

compile .gui file to iOS Storyboard

./ .gui

compile .gui file to HTML/CSS (Bootstrap style)

./ .gui ```


Will pix2code supports other target platforms/languages?

No, pix2code is only a research project and will stay in the state described in the paper for consistency reasons. This project is really just a toy example but you are of course more than welcome to fork the repo and experiment yourself with other target platforms/languages.

Will I be able to use pix2code for my own frontend projects?

No, pix2code is experimental and won't work for your specific use cases.

How is the model performance measured?

The accuracy/error reported in the paper is measured at the DSL level by comparing each generated token with each expected token. Any difference in length between the generated token sequence and the expected token sequence is also counted as error.

How long does it take to train the model?

On a Nvidia Tesla K80 GPU, it takes a little less than 5 hours to optimize the 109 * 10^6 parameters for one dataset; so expect around 15 hours if you want to train the model for the three target platforms.

I am a front-end developer, will I soon lose my job?

(I have genuinely been asked this question multiple times)

TL;DR Not anytime soon will AI replace front-end developers.

Even assuming a mature version of pix2code able to generate GUI code with 100% accuracy for every platforms/languages in the universe, front-enders will still be needed to implement the logic, the interactive parts, the advanced graphics and animations, and all the features users love. The product we are building at Uizard Technologies is intended to bridge the gap between UI/UX designers and front-end developers, not replace any of them. We want to rethink the traditional workflow that too often results in more frustration than innovation. We want designers to be as creative as possible to better serve end users, and developers to dedicate their time programming the core functionality and forget about repetitive tasks such as UI implementation. We believe in a future where AI collaborate with humans, not replace humans.

Media coverage

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.