zach96guan / 11-785_Project

Deep learning final project, combining Mask R-CNN into Cycle-GAN to achieve better accuracy in style transfer.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

11-785 Deep Learning Final Project

Cycle-GAN has achieved great results in style transfer. However, it is sometimes not precise enough. For example, as figure 1 demonstrates, if we want to transfer a horse to zebra, it applies the stripes to not only the horse body, but also the background, even the human riding the horse. Besides, the background of the image is also patternized with the strip and rendering a lower saturation than original one. We suppose Cycle-GAN could render better result if we provide it with a mask of the object that we want to apply style transfer on. It can be applied to resolve the ambiguity in the image for Cycle-GAN to generate more precise transformation. Our team plans to combine the two neural networks, by feeding in the segmentation of the input image (from Mask R-CNN), in addition to the input image, into Cycle-GAN to achieve better accuracy.

About

Deep learning final project, combining Mask R-CNN into Cycle-GAN to achieve better accuracy in style transfer.


Languages

Language:Jupyter Notebook 99.1%Language:Python 0.9%Language:Shell 0.0%Language:MATLAB 0.0%Language:TeX 0.0%Language:Dockerfile 0.0%