Need help with sketch_simplification?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

bobbens
508 Stars 78 Forks Other 14 Commits 7 Opened issues

Description

Models and code related to sketch simplification of rough sketches.

Services available

!
?

Need anything else?

Contributors list

# 21,204
Objective...
C++
C
neuralart 2.4K Java top 10

Sketch Simplification

Example result Example result of a sketch simplification. Image copyrighted by Eisaku Kubonouchi (@EISAKUSAKU) and only non-commercial research usage is allowed.

Overview

This code provides pre-trained models used in the research papers:

   "Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup"
   Edgar Simo-Serra*, Satoshi Iizuka*, Kazuma Sasaki, Hiroshi Ishikawa (* equal contribution)
   ACM Transactions on Graphics (SIGGRAPH), 2016

and

   "Mastering Sketching: Adversarial Augmentation for Structured Prediction"
   Edgar Simo-Serra*, Satoshi Iizuka*, Hiroshi Ishikawa (* equal contribution)
   ACM Transactions on Graphics (TOG), 2018

See our project page for more detailed information.

Dependencies

All packages should be part of a standard PyTorch install. For information on how to install PyTorch please refer to the torch website.

Usage

Before the first usage, the models have to be downloaded with:

bash download_models.sh

Next test the models with:

python simplify.py

You should see a file called

out.png
created with the output of the model.

Application options can be seen with:

python simplify.py --help

Pencil Drawing Generation

Using the same interface it is possible to perform pencil drawing generation. In this case, the input should be a clean line drawing and not a rough sketch, and the line drawings can be generated by:

python simplify.py --img test_line.png --out out_rough.png --model model_pencil2.t7

This will generate a rough version of

test_line.png
as
out_rough.png
. By changing the model it is possible to change the type of rough sketch being generated.

Models

  • model_mse.t7
    : Model trained using only MSE loss (SIGGRAPH 2016 model).
  • model_gan.t7
    : Model trained with MSE and GAN loss using both supervised and unsupervised training data (TOG 2018 model).
  • model_pencil1.t7
    : Model for pencil drawing generation based on artist 1 (dirty and faded pencil lines).
  • model_pencil2.t7
    : Model for pencil drawing generation based on artist 2 (clearer overlaid pencil lines).

Reproducing Paper Figures

For replicability we include code to replicate the figures in the paper. After downloading the models you can run it with:

./figs.sh

This will convert the input images in

figs/
and save the output in
out/
. We note that there are small differences with the results in the paper due to hardware differences and small differences in the torch/pytorch implementations. Furthermore, results are shown without the post-processing mentioned in the notes at the bottom of this document.

Please note that we do not have the copyright for all these images and in general only non-commercial research usage is permitted. In particular,

fig16_eisaku.png
,
fig06_eisaku_robo.png
,
fig06_eisaku_joshi.png
, and
fig01_eisaku.png
are copyright by Eisaku Kubonoichi (@EISAKUSAKU) and only non-commercial research usage is allowed. The images
fig14_pepper.png
and
fig06_pepper.png
are licensed by David Revoy (www.davidrevoy.com) under CC-by 4.0.

Training

Please see the training readme.

Notes

  • Models are in Torch7 format and loaded using the PyTorch legacy code.
  • This was developed and tested on various machines from late 2015 to end of 2016.
  • Provided models are under a non-commercial creative commons license.
  • Post-processing is not performed. You can perform it manually with
    convert out.png bmp:- | mkbitmap - -t 0.3 -o - | potrace --svg --group -t 15 -o - > out.svg
    .

Citing

If you use these models please cite:

@Article{SimoSerraSIGGRAPH2016,
   author    = {Edgar Simo-Serra and Satoshi Iizuka and Kazuma Sasaki and Hiroshi Ishikawa},
   title     = {{Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup}},
   journal   = "ACM Transactions on Graphics (SIGGRAPH)",
   year      = 2016,
   volume    = 35,
   number    = 4,
}

and

@Article{SimoSerraTOG2018,
   author    = {Edgar Simo-Serra and Satoshi Iizuka and Hiroshi Ishikawa},
   title     = {{Mastering Sketching: Adversarial Augmentation for Structured Prediction}},
   journal   = "ACM Transactions on Graphics (TOG)",
   year      = 2018,
   volume    = 37,
   number    = 1,
}

Acknowledgements

This work was partially supported by JST CREST Grant Number JPMJCR14D1 and JST ACT-I Grant Numbers JPMJPR16UD and JPMJPR16U3.

License

This sketch simplification code is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the license for further details.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.