gognlin / DeepLearningAnimePapers

A list of papers and other resources on deep learning with anime style images.

DeepLearningAnimePapers

A list of papers and other resources on computer vision and deep learning with anime style images. Contributions welcome!

Contents

• Anime Datasets
• Anime Papers
  • Anime Colorization
  • Anime Face Recognition
  • Anime Generation
  • Anime Inpainting
  • Anime Image-to-Image Translation
  • Anime Pose Estimation
  • Anime Sketch Editing
  • Anime Style Transfer
  • Anime Misc
  • Anime Non-Deep Learning
• General Papers
  • Image Colorization
  • Image Generation
  • Image Inpainting
• Other Repositories
  • Anime Repositories
  • General Repositories

Anime Datasets

• AnimeDrawingsDataset: 2000 anime/manga images with 2D pose annotations [github] (Last updated June 10 2015)
• Manga109: 109 manga volumes from the 1970s to 2010s [link]
• Nico-Illust: 400,000 images from Niconico Seiga and Niconico Shunga with metadata [link]
• Danbooru2017: 2.9+ million images database from Danbooru with tags [link] (Last updated March 19 2018)
• Danbooru2018: 3.3+ million images database from Danbooru with tags [link] (Last updated April 04 2019)
• MyAnimeList Dataset: crawled data about 14k+ anime, 300k+ users, and 80+ million animelist records [link] (Last Updated June 29 2018)

Anime Papers

Anime Colorization

• Into the Colorful World of Webtoons: Through the Lens of Neural Networks [semanticscholar] (2017)
• Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN [arXiv] (June 11 2017)
• cGAN-based Manga Colorization Using a Single Training Image [arXiv] (June 21 2017)
• Automatic Colorization of Webtoons Using Deep Convolutional Neural Networks [link] (February 2018)
• User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks [arXiv] (August 10 2018)

Anime Face Recognition

• A Multi-Label Convolutional Neural Network for Automatic Image Annotation [link] (July 5 2015)
• Disentangling Style and Content in Anime Illustrations [arXiv] (May 26, 2019)

Anime Generation

• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks [arXiv] [official implementation] (August 18 2017)
• Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks [arXiv] (September 6 2018)
• StyleGAN trained on Danbooru2018 dataset [very detailed description of everything]

Anime Image-to-Image Translation

• Improving Shape Deformation in Unsupervised Image-to-Image Translation [arXiv] (August 13 2018)
• Landmark Assisted CycleGAN for Cartoon Face Generation [arXiv] (July 2 2019)

Anime Inpainting

• Joint Gap Detection and Inpainting of Line Drawings [link] (2017)
• Decensoring Hentai with Deep Neural Networks https://github.com/deeppomf/DeepCreamPy (my project!) based on following Image Inpainting paper [arXiv]

Anime Pose Estimation

• Pose Estimation of Anime/Manga Characters: A Case for Synthetic Data [ACM] (December 4 2016)

Anime Sketch Editing

• Sketch Simplification by Classifying Strokes [IEEE] (December 4 2016)
• Mastering Sketching: Adversarial Augmentation for Structured Prediction [Project page with paper link] (January 18 2018) Note the version of this paper on arXiv is outdated.
• Real-Time Data-Driven Interactive Rough Sketch Inking [Project page with paper link] (January 18 2018)

Anime Style Transfer

• Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN [arXiv] (June 11 2017)
• Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks [arXiv] (May 21 2018)

Anime Super Resolution

• Images Super Resolution waifu2x - Image Super-Resolution for Anime-style art using Deep Convolutional Neural Networks, based on following paper [arXiv]

Anime Misc

• Deep Extraction of Manga Structural Lines [ACM] (July 2017)
• A Survey of Comics Research in Computer Science [arXiv] (April 16 2018)

Anime Non-Deep Learning

• DrawFromDrawings: 2D Drawing Assistance via Stroke Interpolation with a Sketch Database [PubMed] (2016)
• Sketch-based Manga Retrieval using Manga109 Dataset [SpringerLink] (November 9 2016)
• Interactive Region Segmentation for Manga [IEEE] (December 4 2016)
• Face Detection and Face Recognition of Cartoon Characters Using Feature Extraction [IIEEJ] (2012)

General Papers

Image Colorization

• Real-Time User-Guided Image Colorization with Learned Deep Priors [arXiv] (May 8 2017)

Image Generation

• StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks [arXiv] (October 19 2017) [original pytorch implementation]
• Progressive Growing of GANs for Improved Quality, Stability, and Variation [arXiv] (October 27 2017) [original theano/lasagne implementation]

Image Inpainting

Title	Contributions	Shortcomings	Maximum Input Size	Code?
Context Encoders: Feature Learning by Inpainting [arXiv] (April 25 2016)	First use of CNNs in image inpainting. Utilizes an adversarial loss	Completed regions blurry.	227 x 227 x 3 for random region and 128 x 128 x 3 for center region	[official torch implementation]
Semantic Image Inpainting with Deep Generative Models [arXiv] (July 26 2016)	Missing content infered by searching for closest encoding of the corrupted image in the latent image manifold.	No end to end training. IMHO, generating images is harder than inpainting images because with inpainting, there is always ground truth present. So converting inpainting to the harder problem of generating images might not be the way to go.	64 x 64 x 3, arbitrary mask	[tensorflow implementation]
Globally and Locally Consistent Image Completion [link] (2017)	Dilated convolutions 2 discriminators: one local discriminator for the completed region and one global discriminator for whole image	Long training time. Poisson blending needed. Complex training process. Completion network is trained, then the completion network is fixed and discriminators are trained, then finally both are trained.	256 x 256 x 3	[official torch implementation. no training code] [tensorflow implementation. missing gan loss] [tensorflow implementation. has UI]
Image Inpainting using Multi-Scale Feature Image Translation [arXiv] (November 23 2017)	2 stages: coarse prediction and refinement through feature based texture swapping Framework can be adapted to multi-scale		256 x 256 x 3 input, arbitrary mask	Soon
Context-Aware Semantic Inpainting [arXiv] (December 21 2017)
Light-weight pixel context encoders for image inpainting [arXiv] (January 17 2018)
High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks [arXiv] (January 23 2018)	Conditioned on facial attributes. Progressive growing of GANs. Three new losses: attribute, feature, and boundary.	Fails to learn low level skin features. Long training time.	1024 x 1024 x 3, arbitrary mask	Soon
Deep Structured Energy-Based Image Inpainting [arXiv] (January 24 2018)
Image Inpainting for Irregular Holes Using Partial Convolutions [arXiv] (April 20 2018)	Introduces partial convolutions, which exclude information from the mask. No post processing.		512 x 512 x 3, arbitrary mask
Free-Form Image Inpainting with Gated Convolution [arXiv] (June 10 2018)	Utilizes gated convolutions. State of the art inpainting for irregular masks. No post processing.		512 x 512 x 3, arbitrary mask	Soon

Other Repositories

Anime Repositories

• none

General Repositories

• 3D Machine Learning
• Awesome Deep Vision