Need help with style-swap?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

rtqichen
380 Stars 92 Forks MIT License 29 Commits 5 Opened issues

Description

code for "Fast Patch-based Style Transfer of Arbitrary Style".

Services available

!
?

Need anything else?

Contributors list

# 43,161
Lua
Shell
Python
bayesia...
15 commits
# 13,076
Lua
Jupyter...
cyclega...
MATLAB
5 commits
# 3,976
Lua
MATLAB
Jupyter...
cyclega...
1 commit

Fast Patch-based Style Transfer of Arbitrary Style

Paper: https://arxiv.org/abs/1612.04337

Code is written in Torch. CUDA and CPU modes are available.

Examples

(3x3 Patches) Content - w/ Starry Night - w/ Small Worlds I

with AvgPooling - using Inverse Network - using Inverse Network

(w/ Composition X) Original - 5x5 Patch - 9x9 Patch - 15x15 Patch

(w/ La Muse) Original - 3x3 Patch - 5x5 Patch - 9x9 Patch

Download Pretrained VGG-19

git clone https://github.com/rtqichen/style-swap
cd style-swap/models
sh download_models.sh
cd ..

Usage

Stylizing a single image:

th style-swap.lua --content images/content/bike.jpg --style images/style/starry_night.jpg

More options:

th style-swap.lua --help

eg. increase

--patchSize
for more abstract stylization
th style-swap.lua --content images/content/brad_pitt.jpg --style images/style/la_muse.jpg --patchSize 7 --patchStride 3

eg. use

--contentBatch
to stylize all images in a directory.
th style-swap.lua --contentBatch images/content --style images/style/starry_night.jpg

Training an inverse network

Install nninit module:

luarocks install nninit

Train:

th train-vgg-decoder.lua --contentDir /path/to/dir --styleDir /path/to/dir

More options:

th train-vgg-decoder.lua --help

For training the network in our paper, we used images from MS COCO and the Painter by Numbers competition hosted by Kaggle. A trained network can be downloaded here.

Video

Frame-by-frame stylization can be done using the

-contentBatch
option.

An example script using

ffmpeg
to extract frames, stylize, and re-encode a video.
mkdir video_frames
ffmpeg -i /path/to/video -qscale:v 2 video_frames/video_%04d.jpg
th style-swap --contentBatch video_frames --style /path/to/style/file --save stylized_frames
ffmpeg -i stylized_frames/video_%04d_stylized.jpg -c:v libx264 -pix_fmt yuv420p stylized_video.mp4

Examples of stylized videos are placed in the videos folder. (Original video by TimeLapseHD.)

Reducing GPU Memory Usage

A few ways to reduce memory usage for

style-swap.lua
: - Decrease
--maxStyleSize
and
--maxContentSize
. The latter changes the size of the resulting image. - Increase
--patchStride
. This extracts less patches to use for style swap. Best to use a larger
--patchSize
to ensure the patches still overlap. - Last resort: use CPU-only mode by specifying
--cpu
.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.