README.md
1<!--- Licensed to the Apache Software Foundation (ASF) under one -->
2<!--- or more contributor license agreements. See the NOTICE file -->
3<!--- distributed with this work for additional information -->
4<!--- regarding copyright ownership. The ASF licenses this file -->
5<!--- to you under the Apache License, Version 2.0 (the -->
6<!--- "License"); you may not use this file except in compliance -->
7<!--- with the License. You may obtain a copy of the License at -->
8
9<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
10
11<!--- Unless required by applicable law or agreed to in writing, -->
12<!--- software distributed under the License is distributed on an -->
13<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
14<!--- KIND, either express or implied. See the License for the -->
15<!--- specific language governing permissions and limitations -->
16<!--- under the License. -->
17
18# MXNet-Gluon-Style-Transfer
19
20This repo provides MXNet Implementation of **[Neural Style Transfer](#neural-style)** and **[MSG-Net](#real-time-style-transfer)**.
21
22**Tabe of content**
23
24* [Slow Neural Style Transfer](#neural-style)
25* [Real-time Style Transfer](#real-time-style-transfer)
26 - [Stylize Images using Pre-trained MSG-Net](#stylize-images-using-pre-trained-msg-net)
27 - [Train Your Own MSG-Net Model](#train-your-own-msg-net-model)
28
29## Neural Style
30
31[A Neural Algorithm of Artistic Style](https://arxiv.org/abs/1508.06576) by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.
32
33
34**Download the images**
35
36```bash
37python download_images.py
38```
39
40**Neural style transfer**
41
42```bash
43python main.py optim --content-image images/content/venice-boat.jpg --style-image images/styles/candy.jpg
44```
45* `--content-image`: path to content image.
46* `--style-image`: path to style image.
47* `--output-image`: path for saving the output image.
48* `--content-size`: the content image size to test on.
49* `--style-size`: the style image size to test on.
50* `--cuda`: set it to 1 for running on GPU, 0 for CPU.
51
52<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g1.jpg" width="260px" /> <img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g2.jpg" width="260px" />
53<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g3.jpg" width="260px" />
54<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g4.jpg" width="260px" />
55<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g5.jpg" width="260px" />
56<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g6.jpg" width="260px" />
57<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g7.jpg" width="260px" />
58<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g8.jpg" width="260px" />
59<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/g9.jpg" width="260px" />
60
61## Real-time Style Transfer
62<table width="100%" border="0" cellspacing="15" cellpadding="0">
63 <tbody>
64 <tr>
65 <td>
66 <b>Multi-style Generative Network for Real-time Transfer</b> [<a href="https://arxiv.org/pdf/1703.06953.pdf">arXiv</a>] [<a href="http://computervisionrutgers.github.io/MSG-Net/">project</a>] <br>
67 <a href="http://hangzh.com/">Hang Zhang</a>, <a href="http://eceweb1.rutgers.edu/vision/dana.html">Kristin Dana</a>
68<pre>
69@article{zhang2017multistyle,
70 title={Multi-style Generative Network for Real-time Transfer},
71 author={Zhang, Hang and Dana, Kristin},
72 journal={arXiv preprint arXiv:1703.06953},
73 year={2017}
74}
75</pre>
76 </td>
77 <td width="440"><a><img src ="https://raw.githubusercontent.com/zhanghang1989/MSG-Net/master/images/figure1.jpg" width="420px" border="1"></a></td>
78 </tr>
79 </tbody>
80</table>
81
82
83### Stylize Images Using Pre-trained MSG-Net
840. Download the images and pre-trained model
85 ```bash
86 python download_images.py
87 python models/download_model.py
88 ```
890. Test the model
90 ```bash
91 python main.py eval --content-image images/content/venice-boat.jpg --style-image images/styles/candy.jpg --model models/21styles.params --content-size 1024
92 ```
93* If you don't have a GPU, simply set `--cuda=0`. For a different style, set `--style-image path/to/style`.
94 If you would to stylize your own photo, change the `--content-image path/to/your/photo`.
95 More options:
96
97 * `--content-image`: path to content image you want to stylize.
98 * `--style-image`: path to style image (typically covered during the training).
99 * `--model`: path to the pre-trained model to be used for stylizing the image.
100 * `--output-image`: path for saving the output image.
101 * `--content-size`: the content image size to test on.
102 * `--cuda`: set it to 1 for running on GPU, 0 for CPU.
103
104<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/1.jpg" width="260px" /> <img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/2.jpg" width="260px" />
105<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/3.jpg" width="260px" />
106<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/4.jpg" width="260px" />
107<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/5.jpg" width="260px" />
108<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/6.jpg" width="260px" />
109<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/7.jpg" width="260px" />
110<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/8.jpg" width="260px" />
111<img src ="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/style_transfer/images/9.jpg" width="260px" />
112
113### Train Your Own MSG-Net Model
1140. Download the style images and COCO dataset
115Note: Dataset from [COCO 2014](http://cocodataset.org/#download).
116The dataset annotations and site are Copyright COCO Consortium and licensed CC BY 4.0 Attribution.
117The images within the dataset are available under the Flickr Terms of Use.
118See original [dataset source](http://cocodataset.org/#termsofuse) for details
119 ```bash
120 python download_images.py
121 python dataset/download_dataset.py
122 ```
1230. Train the model
124 ```bash
125 python main.py train --epochs 4
126 ```
127* If you would like to customize styles, set `--style-folder path/to/your/styles`. More options:
128 * `--style-folder`: path to the folder style images.
129 * `--vgg-model-dir`: path to folder where the vgg model will be downloaded.
130 * `--save-model-dir`: path to folder where trained model will be saved.
131 * `--cuda`: set it to 1 for running on GPU, 0 for CPU.
132
133
134The code is mainly modified from [PyTorch-Style-Transfer](https://github.com/zhanghang1989/PyTorch-Style-Transfer).
135
