IShengFang / TypographyResearchCollection

The research collection of typography

Typography Research Collection

[hackmd][github][github pages]

Typography is the cross between technology and liberal arts. This page is a research collection that includes computer graphics, computer vision, machine learning that related to typography.

Visual Text Generation and Editing

• Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models

  • Bin Fu, Fanghua Yu, Anran Liu, Zixuan Wang, Jie Wen, Junjun He, Yu Qiao
  • CVPR 2024

• VecFusion: Vector Font Generation with Diffusion

  • Vikas Thamizharasan, Difan Liu, Shantanu Agarwal, Matthew Fisher, Michael Gharbi, Oliver Wang, Alec Jacobson, Evangelos Kalogerakis
  • CVPR 2024
  • [arxiv]

• FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning
  • Zhenhua Yang, Dezhi Peng, Yuxin Kong, Yuyi Zhang, Cong Yao, Lianwen Jin
  • AAAI 2024
  • [project page][arxiv][code][demo]

• AnyText: Multilingual Visual Text Generation And Editing
  • Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng∗, Xuansong Xie
  • ICLR 2024
  • [paper][code][demo]

• GlyphControl: Glyph Conditional Control for Visual Text Generation
  • Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, Kai Chen
  • NeurIPS 2023
  • [paper][code]

• DiffUTE: Universal Text Editing Diffusion Model
  • Haoxing Chen, Zhuoer Xu1, Zhangxuan Gu*, Jun Lan, Xing Zheng, Yaohui Li, Changhua Meng, Huijia Zhu, Weiqiang Wang
    • ∗Corresponding author
  • NeurIPS 2023
  • [paper][code]

• TextDiffuser: Diffusion Models as Text Painters
  • Jingye Chen*, Yupan Huang*, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei
    • *Equal Contribution
  • NeurIPS 2023
  • [project page][paper][code][demo][colab]

• GlyphDraw: Learning to Draw Chinese Characters in Image Synthesis Models Coherently
  • Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, Xiaodong Lin
  • arxiv 2023
  • [project page][paper][code]

• DeepFloyd IF
  • Inspired by "Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding", NeurIPS2022
    • aka Imagen
  • [code]

• Character-Aware Models Improve Visual Text Rendering
  • arxiv
  • Rosanne Liu∗, Dan Garrette∗, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, Noah Constant∗
    • *Equal contribution
  • [paper]

• OCR-VQGAN: Taming Text-within-Image Generation
  • Juan A. Rodríguez, David Vázquez, Issam Laradji, Marco Pedersoli, Pau Rodríguez
  • WACV2023
  • [paper(CVF)][paper(arxiv)][code]

Font Stye Transfer and Glyph Generation

NN Approach

• DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
  • Maham Tanveer, Yizhi Wang, Ali Mahdavi-Amiri, Hao Zhang
  • ICCV2023
  • [project page][paper][code] [demo]

• Word-As-Image for Semantic Typography
  • Shir Iluz*, Yael Vinker*, Amir Hertz, Daniel Berio, Daniel Cohen-Or, Ariel Shamir
    • *Denotes equal contribution
  • SIGGRAPH 2023 - Honorable Mention Award
  • [project page][paper][code][demo]

• DeepVecFont-v2: Exploiting Transformers to Synthesize Vector Fonts with Higher Quality
  • Yuqing Wang, Yizhi Wang, Longhui Yu, Yuesheng Zhu, Zhouhui Lian
  • CVPR 2023
  • [paper(CVPR)][code]

• DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning
  • Yizhi Wang, Zhouhui Lian
  • SIGGRAPH Asia 2021
  • [project page][code][paper]

• StrokeStyles: Stroke-based Segmentation and Stylization of Fonts

  • Daniel Berio, Frederic Fol Leymarie, Paul Asente, Jose Echevarria
  • ACM Transactions on GraphicsVolume 41Issue 3Article No.: 28pp 1–21
  • [paper]

• DG-Font: Deformable Generative Networks for Unsupervised Font Generation

  • Yangchen Xie, Xinyuan Chen, Li Sun, Yue Lu
  • [paper(CVPR)][paper(arxiv)][code]
  • CVPR2021

• Multiple Heads are Better than One:Few-shot Font Generation with Multiple Localized Experts
  • Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim
  • [paper][code]
  • ICCV 2021

• Font Style that Fits an Image -- Font Generation Based on Image Context
  • [paper][code]
  • Taiga Miyazono, Brian Kenji Iwana, Daichi Haraguchi, and Seiichi Uchida
  • arxiv 2021

• Few-shot Font Generation with Localized Style Representations and Factorization
  • Song Park*, Sanghyuk Chun*, Junbum Cha, Bado Lee, Hyunjung Shim1
    • *: equal contribution
  • [project page] [paper][code]
  • AAAI 2021

• Handwritten Chinese Font Generation with Collaborative Stroke Refinement
  • [paper][WACV2021][code(dead)]
  • Chuan Wen, Jie Chang, Ya Zhang, Siheng Chen, Yanfeng Wang, Mei Han, Qi Tian
  • WACV 2021

• RD-GAN: Few/Zero-Shot Chinese Character Style Transfer via Radical Decomposition and Rendering
  • Yaoxiong Huang, Mengchao He, Lianwen Jin, Yongpan Wang
  • [paper(ECVA)]
  • ECCV 2020

• Few-shot Compositional Font Generation with Dual Memory
  • Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, Hwalsuk Lee.
  • [paper] [code]
  • ECCV 2020

• CalliGAN: Style and Structure-aware Chinese Calligraphy Character Generator
  • [paper][code]
  • Shan-Jean Wu, Chih-Yuan Yang and Jane Yung-jen Hsu
  • AI for Content Creation Workshop CVPR 2020.

• A Learned Representation for Scalable Vector Graphics
  • [paper]
  • Raphael Gontijo Lopes, David Ha, Douglas Eck, Jonathon Shlens
  • ICCV 2019
  • ICLR workshop 2019

• Large-scale Tag-based Font Retrieval with Generative Feature Learning

  • [paper][page]
  • Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, Jiebo Luo
  • ICCV 2019

• DynTypo: Example-based Dynamic Text Effects Transfer

  • [paper]
  • Yifang Men, Zhouhui Lian, Yingmin Tang, Jianguo Xiao
  • CVPR 2019
  • <iframe width="560" height="315" src="https://www.youtube.com/embed/FkFQ6bV1s-o" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

• Typography with Decor: Intelligent Text Style Transfer

  • Wenjing Wang, Jiaying Liu, Shuai Yang, and Zongming Guo
  • CVPR2019
  • [project page] [paper] [code]

  

• DeepGlyph

  • [websites]
  • <iframe width="560" height="315" src="https://www.youtube.com/embed/T70k-0qgccs" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

• Coconditional Autoencoding Adversarial Networks for Chinese Font Feature Learning

  • [paper]
  • arxiv 2018
  • Zhizhan Zheng, Feiyun Zhang

• TET-GAN: Text Effects Transfer via Stylization and Destylization

  • [paper]
  • Shuai Yang, Jiaying Liu, Wenjing Wang, Zongming Guo
  • AAAI2019

• SCFont: Structure-Guided Chinese Font Generation via Deep Stacked Networks
  • [paper]
  • Yue Jiang, Zhouhui Lian*, Yingmin Tang, Jianguo Xiao
  • AAAI 2019

• Separating Style and Content for Generalized Style Transfer

  • [paper][code]
  • Yexun Zhang, Ya Zhang, Wenbin Cai
  • CVPR 2018
  • Networks
  • results

• Deep Learning for Classical Japanese Literature

  • [paper][GitHub]
  • Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, David Ha
  • NeurIPS 2018
  • Kuzushiji-MNIST,Kuzushiji-49 and Kuzushiji-Kanji
  • Kuzushiji-MNIST
  • Kuzushiji-Kanji

• Multi-Content GAN for Few-Shot Font Style Transfer

  • [code][paper][blog]
  • Azadi, Samaneh, Matthew Fisher, Vladimir Kim, Zhaowen Wang, Eli Shechtman, and Trevor Darrell.
  • CVPR2018

• Learning to Write Stylized Chinese Characters by Reading a Handful of Examples

  • [paper]
  • Danyang Sun∗, Tongzheng Ren∗, Chongxuan Li, Hang Su†, Jun Zhu†
  • IJCAI 2018
  • SA-VAE

• DCFont: An End-To-End Deep Chinese Font Generation System
  • [paper]
  • Juncheng Liu, Zhouhui Lian, Jianguo Xiao
  • SIGGRAPH Asia 2017

• zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks

  • [blog] [code(TensorFlow)] [code(pytorch)(xuan-li)] [code(pytorch)( EuphoriaYan)]
  • Yuchen Tian
  • 2017

  

• Rewrite: Neural Style Transfer For Chinese Fonts

  • [code]
  • Yuchen Tian
  • 2016

  

• Automatic generation of large-scale handwriting fonts via style learning

  • [paper]
  • Zhouhui Lian, Bo Zhao, and Jianguo Xiao
  • SIGGRAPH ASIA 2016

• A Book from the Sky 天书: Exploring the Latent Space of Chinese Handwriting
  • [blog]
  • Gene Kogan

• Letter Spirit: An Emergent Model of the Perception and Creation of Alphabetic Style
  • [paper]
  • Douglas Hofstadter, Gary McGraw
  • 1993

Other Approach

• Automatic Generation of Typographic Font from a Small Font Subset

  • [paper]
  • Tomo Miyazaki, Tatsunori Tsuchiya, Yoshihiro Sugaya, Shinichiro Omachi, Masakazu Iwamura, Seiichi Uchida, Koichi Kise
  • 2017

  

• FlexyFont: Learning Transferring Rules for Flexible Typeface Synthesis

  • [paper]
  • H. Q. Phan, H. Fu, and A. B. Chan
  • 2015 Computer Graphics Forum

  

• Awesome Typography: Statistics-Based Text Effects Transfer

  • [paper][code]
  • Shuai Yang, Jiaying Liu, Zhouhui Lian and Zongming Guo
  • CVPR 2017

  

• Easy generation of personal Chinese handwritten fonts

  • [paper]
  • Baoyao Zhou, Weihong Wang, and Zhanghui Chen
  • 2011 ICME(IEEE International Conference on Multimedia and Expo)

  

• Automatic shape morphing for Chinese characters

  • [paper]
  • Zhouhui Lian, and Zhouhui Lian
  • SIGGRAPH Asia 2012
  • 

Struture Learning

• Analyzing 50k fonts using deep neural networks

  • [blog]
  • Erik Bernhardsson
  • 2016

  

• Learning a Manifold of Fonts

  • [paper][project(demo)]
  • Neill D.F. Campbell, and Jan Kautz
  • SIGGRAPH 2014

  

DataSet

• Chinese Handwriting Recognition Competition 2013
  • [paper][download]

• Deep Learning for Classical Japanese Literature
  • [paper][GitHub]
  • Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, David Ha
  • NeurIPS 2018
  • Kuzushiji-MNIST,Kuzushiji-49 and Kuzushiji-Kanji
  • Kuzushiji-MNIST
  • Kuzushiji-Kanji

Other Application

• FontCode: Embedding Information in Text Documents Using Glyph Perturbation
  • [paper][project page]
  • Chang Xiao, Cheng Zhang, Changxi Zheng
  • SIGGRAPH 2018
  • <iframe width="560" height="315" src="https://www.youtube.com/embed/dejrBf9jW24" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>