WBC

Mini-Project/Files/NOTES.txt

@@ -0,0 +1,28 @@
+As we can see, GANs have a very impressive results and realistic content. It can also be modelled for other applications like image restoration
+
+But it is very difficult to train it. As the outputs are realistic, not real due to the loss function.
+
+As you may know loss function helps to evaluate the algorithm and models the dataset.
+
+If the predictions are totally off, then the loss function will output a higher number. If they're pretty good, it'll output a lower number.
+
+
+In Surface Representation we used Edge Preserving Filtering. It is basically an image processing technique.
+
+It helps to reduce noise and smoothens image while retaining the texture and sharp edges.
+
+
+In Structure representation 
+
+We at first use felzenszwalb algorithm to segment images into separate regions. 
+
+As superpixel algorithms only consider the similarity of pixels and take average colour value 
+
+We further introduce selective search to merge segmented regions
+
+Finally, we used adaptive colouring Algorithm as it enhances the contrast and reduces hazing effect
+
+
+In texture representation, we used random colour shift algorithm as it removes unnecessary colour information and generates random intensity map with luminance
+
+

Mini-Project/Files/USE CASE.drawio

@@ -0,0 +1 @@
+<mxfile host="app.diagrams.net" modified="2021-04-30T07:03:20.623Z" agent="5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36" etag="GGFh87R5P-jFz96TWxun" version="14.6.9" type="google"><diagram id="KfFczl9QfsM-k7m8ZQHQ" name="Page-1">7Ztdc5s4FIZ/jS+XAYkvXzp203aanUk309m2d4pRbKYCsVj+SH/9SiAwSPKaZg1OptzY6CBkSe/Dkc4BT+A8ObzPUbb+k0aYTIAdHSZwMQHAsWHIv4TlubT4TlAaVnkcyUpHw0P8E1dXSus2jvCmVZFRSlictY1LmqZ4yVo2lOd03672REn7VzO0wprhYYmIbv07jti6tIYgONo/4Hi1rn7Z8aflmQRVleVINmsU0X3DBN9N4DynlJVHyWGOiZi8al7K625PnK07luOUdbnA/uvzPzc/Pt09Tf00+/5z79+R3R+ylR0iWzngLxucyw6z52oWeN8zcbhNyGzJaD6BNzucs5jP0x16xOSebmIW05RXeaSM0aRRYUbilTjBaMata5YQXnD4Id0yEqd4Xitnc+MTTdktSmIieLm/m32Ttge6zZeiC2vGuPTAgzP+wQcrPkSFjbWidEUwyuKNteQ9ECeWm6Lq7VPZIj9stVnS5ri8LOeB9xkfTk6wU8vGecc0wSznjdryAl8KLUl3ZXF/xAZK07pBTHUVkqCu6naPWvIDKecvSAs0aRd4hwnNRn0vom99Y19LYKgJrOnKW+GOkhdu9uuY4YcMFXO85766rdUblgXYjuX5LWmgrowDDNKEfUnjjtIULtC1z+riDamLpy93GaEomgg6fMJ//uaRe0d/JY4+JmJvoOrGZ4O1BdqwnP7gbo4Iv7lIaSpEfYoJUUxIOsolnz2cGzxoEkcROUVETrdphKO370b5Hkm9X30DF7aBC9AXF77GxRzlfJ+ZiqGb2NgW2GABTiww2YycXNx5eMp+ytMhcQ2QeH1BEnRw6mk0E0HHUcwIbdaFGk4bBnyI2dfG8TchmOXJ0uIg9SsKz1cWkwuWP3+teiQKjd6K4rG7Ranqbzk9ONIiLAUBPoXVCM7sdRjKV5j9Rz3XjFSDGdN6U9lyTBCLd+3umjiSv3BPYz6Qmlg4VZc7BcVymPKqZqimNOTCdkOB0k45DVo7BdX1qF8OemgA3UeJ8C6FG2yX4nRJtiJGr6yrVh1ZGr3jZb0jVLZW3rWX0Om45S3mFwaW11IGOLWlIc7U07XxvZ60qZJpoziKg3ZN0jiO4b5RXfDltNFTcL+pNuqN404N2tjAGvTO0bNos8gcLi4QQ+M6d3Es3NAKFIcKgeGuDQxYwN6w0HNvX7IIMXOwOJLRj8PwrKnb9ueBYaENreombqLh9IaGnvtbYIILNEYQegHBU1cOz9Y58I0Lh9sbBnqqUV/WtWxBM0HwwqD7ukJ2CfbPBvFTs9zDBPF11FStNmo+qXMQrzTkOsNG8Y6e1PxFAmWKqspDveYUVXf6zqaaHPua+EE166k+PuyK31mO+8bvRenS0QEeY8LRA/5fBE2JzN8wtPZCoAjK98RQ2yFxu86UA+p90+X3SHrOcE5TMVRu/MD7wL8+b/kul4nBfJTP2WxOC+Vfxyd0UfMsSkXxAe3GR7eXfwPGAZavsBRCy3U1lhwfWqaIPHCsMOgJJ9Aly9n9Ad0Ll5+39GDvgrumadelLXxVuyv3Qktb/ehvoKUNdMkaj6z3GSGMsA8GOxj3ccXiaQdtHa79miPQ0+Dlxmt8m24wJkCbCcN7AKEBCdUVXA4J06uv48sjrwsaFygRvvk5+KDvjwBTwnwE53WDA33b8vVMwsDkmBLdp8nhDPCd8QjOwP+deWWvq4ExPX0q/jkb1kCz1NfJTqsvZnUNauoLqzVw6KCmS3J6jOD7ieDdjqSXmlwLdaCuti8N4PtjnReP/2Auqx//Bw7f/Qs=</diagram></mxfile>

Mini-Project/WBC/README.md

@@ -0,0 +1,112 @@
+<img src='paper/shinjuku.jpg' align="left" width=1000>
+
+<br><br><br>
+
+# [CVPR2020]Learning to Cartoonize Using White-box Cartoon Representations
+[project page](https://systemerrorwang.github.io/White-box-Cartoonization/) |   [paper](https://github.com/SystemErrorWang/White-box-Cartoonization/blob/master/paper/06791.pdf) |   [twitter](https://twitter.com/IlIIlIIIllIllII/status/1243108510423896065) |   [zhihu](https://zhuanlan.zhihu.com/p/117422157) |   [bilibili](https://www.bilibili.com/video/av56708333) |  [facial model](https://github.com/SystemErrorWang/FacialCartoonization)
+
+- Tensorflow implementation for CVPR2020 paper “Learning to Cartoonize Using White-box Cartoon Representations”.
+- Improved method for facial images are now available:
+- https://github.com/SystemErrorWang/FacialCartoonization
+
+<img src="images/method.jpg" width="1000px"/>
+<img src="images/use_cases.jpg" width="1000px"/>
+
+## Use cases
+
+### Scenery
+<img src="images/city1.jpg" width="1000px"/>
+<img src="images/city2.jpg" width="1000px"/>
+
+### Food
+<img src="images/food.jpg" width="1000px"/>
+
+### Indoor Scenes
+<img src="images/home.jpg" width="1000px"/>
+
+### People
+<img src="images/person1.jpg" width="1000px"/>
+<img src="images/person2.jpg" width="1000px"/>
+
+### More Images Are Shown In The Supplementary Materials
+
+
+## Online demo
+
+- Some kind people made online demo for this project
+- Demo link: https://cartoonize-lkqov62dia-de.a.run.app/cartoonize
+- Code: https://github.com/experience-ml/cartoonize
+- Sample Demo: https://www.youtube.com/watch?v=GqduSLcmhto&feature=emb_title
+
+## Prerequisites
+
+- Training code: Linux or Windows
+- NVIDIA GPU + CUDA CuDNN for performance
+- Inference code: Linux, Windows and MacOS
+
+
+## How To Use
+
+### Installation
+
+- Assume you already have NVIDIA GPU and CUDA CuDNN installed 
+- Install tensorflow-gpu, we tested 1.12.0 and 1.13.0rc0 
+- Install scikit-image==0.14.5, other versions may cause problems
+
+
+### Inference with Pre-trained Model
+
+- Store test images in /test_code/test_images
+- Run /test_code/cartoonize.py
+- Results will be saved in /test_code/cartoonized_images
+
+
+### Train
+
+- Place your training data in corresponding folders in /dataset 
+- Run pretrain.py, results will be saved in /pretrain folder
+- Run train.py, results will be saved in /train_cartoon folder
+- Codes are cleaned from production environment and untested
+- There may be minor problems but should be easy to resolve
+- Pretrained VGG_19 model can be found at following url:
+https://drive.google.com/file/d/1j0jDENjdwxCDb36meP6-u5xDBzmKBOjJ/view?usp=sharing
+
+
+
+### Datasets
+
+- Due to copyright issues, we cannot provide cartoon images used for training
+- However, these training datasets are easy to prepare
+- Scenery images are collected from Shinkai Makoto, Miyazaki Hayao and Hosoda Mamoru films
+- Clip films into frames and random crop and resize to 256x256
+- Portrait images are from Kyoto animations and PA Works
+- We use this repo(https://github.com/nagadomi/lbpcascade_animeface) to detect facial areas
+- Manual data cleaning will greatly increace both datasets quality
+
+## Acknowledgement
+
+We are grateful for the help from Lvmin Zhang and Style2Paints Research
+
+## License
+- Copyright (C) Xinrui Wang All rights reserved. Licensed under the CC BY-NC-SA 4.0 
+- license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
+- Commercial application is prohibited, please remain this license if you clone this repo
+
+## Citation
+
+If you use this code for your research, please cite our [paper](https://systemerrorwang.github.io/White-box-Cartoonization/):
+
+@InProceedings{Wang_2020_CVPR,
+author = {Wang, Xinrui and Yu, Jinze},
+title = {Learning to Cartoonize Using White-Box Cartoon Representations},
+booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+month = {June},
+year = {2020}
+}
+
+
+# 中文社区
+
+我们有一个除了技术什么东西都聊的以技术交流为主的宇宙超一流二次元相关技术交流吹水群“纸片协会”。如果你一次加群失败，可以多次尝试。
+
+    纸片协会总舵：184467946

Mini-Project/WBC/STEPS.txt

@@ -0,0 +1,17 @@
+conda create -n wbc python=3.6
+
+cd "path"
+
+conda activate wbc
+
+pip install tensorflow==1.12.0
+
+pip install scikit-image==0.14.5
+
+pip install ffmpeg
+
+pip install opencv-python
+
+pip install tqdm
+
+python cartoonize.py

Mini-Project/WBC/index.html

@@ -0,0 +1,76 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+	<meta name="viewport" content="width=device-width, initial-scale=1.0">
+	<title>Learning to Cartoonize Using White-box Cartoon Representations</title>
+	<link rel="stylesheet" type="text/css" href="./index_files/pixl-bk.css">
+	<link rel="stylesheet" type="text/css" href="./index_files/pixl-fonts.css">
+</head>
+
+<body>
+
+<div class="crumb">
+	<a href="https://github.com/lllyasviel">Style2Paints Research</a> →
+	[Wang et al. 2020]
+	</span>
+</div>
+
+
+<div class="content">
+<div class="paperheader">
+  <div class="papertitle"> Learning to Cartoonize Using White-box Cartoon Representations </div>
+  <br>
+  <div class="pubinfo"> Computer Vision and Pattern Recognition (CVPR), June 2020 </div>
+  <br>
+  <div class="authors"> <a href="https://github.com/SystemErrorWang">Xinrui Wang</a> and Jinze Yu </div>
+</div>
+
+<div class="paperimg"><img src="./paper/shinjuku.jpg"></div>
+<div class="longcaption">Example of image cartoonization with our method: left is a frame in the animation "Garden of words", right is a real-world photo processed by our proposed method.</div>
+<div class="header">Abstract</div>
+<p>
+</p><div class="abstract">
+		This paper presents an approach for image cartooniza- tion. By observing the cartoon painting behavior and consulting artists, we propose to separately identify three white-box representations from images: the surface rep- resentation that contains a smooth surface of cartoon im- ages, the structure representation that refers to the sparse color-blocks and flatten global content in the celluloid style workflow, and the texture representation that reflects high- frequency texture, contours, and details in cartoon im- ages. A Generative Adversarial Network (GAN) framework is used to learn the extracted representations and to car- toonize images.
+		<br>
+		The learning objectives of our method are separately based on each extracted representations, making our frame- work controllable and adjustable. This enables our ap- proach to meet artists’ requirements in different styles and diverse use cases. Qualitative comparisons and quanti- tative analyses, as well as user studies, have been con- ducted to validate the effectiveness of this approach, and our method outperforms previous methods in all compar- isons. Finally, the ablation study demonstrates the influence of each component in our framework.
+
+</div>
+<div class="header">Files</div>
+<ul>
+<li> <a href="./paper/06791.pdf">Paper</a> (9 MB PDF)</li>
+<li> <a href="./paper/06791-supp.pdf">Supplementary Material</a> (15 MB PDF)</li>
+</ul>
+<div class="header">See Also</div>
+<ul>
+
+<li> <a href="https://github.com/SystemErrorWang/White-box-Cartoonization">Source Code</a> - Only inference code available now, training code will be updated later.</li>
+
+<li> <a href="https://www.bilibili.com/video/av56708333">Demo Video</a> - Generated with early version of our work in bilibili.com.</li>
+
+
+</ul>
+
+<div class="header">Citation</div>
+<p>
+Xinrui Wang and Jinze Yu<br>
+"Learning to Cartoonize Using White-box Cartoon Representations."<br>
+<i>IEEE Conference on Computer Vision and Pattern Recognition</i>, June 2020.
+
+<!--
+</p><div class="header">BibTeX</div>
+<p>
+</p><pre>@article{Zhang2020,
+   author = "Lvmin Zhang and Edgar Simo-Serra and Yi Ji and Chunping Liu",
+   title = "Generating Digital Painting Lighting Effects via RGB-space Geometry",
+   journal = "ACM Transactions on Graphics",
+   year = "2020"
+}
+</pre>
+</div>
+-->
+
+
+
+</body></html>