CAMS: Color-Aware Multi-Style Transfer

Mahmoud Afifi¹, Abdullah Abuolaim*¹, Mostafa Hussien*², Marcus A. Brubaker¹, Michael S. Brown¹

¹York University
²École de technologie supérieure

* denotes equal contribution

Reference code for the paper CAMS: Color-Aware Multi-Style Transfer. Mahmoud Afifi, Abdullah Abuolaim, Mostafa Hussien, Marcus A. Brubaker, and Michael S. Brown. arXiv preprint, 2021. If you use this code, please cite our paper:

@article{afifi2021coloraware,
  title={CAMS: Color-Aware Multi-Style Transfer},
  author={Afifi, Mahmoud and Abuolaim, Abdullah and Hussien, Mostafa and Brubaker, Marcus A. and Brown, Michael S.},
  journal={arXiv preprint arXiv:2106.13920},
  year={2021}
}

Get Started

Run color_aware_st.py or check the Colab link from here.

To compute the color-aware loss between two images, see test_cams_loss.py. To report the average color-aware loss for a set of pair images, use report_losses_of_image_dir.py.

Manual Selection

Our method allows the user to manually select the color correspondences between palettes or ignore some colors when optimizing.

To enable this mode, use SELECT_MATCHES = True.

Other useful parameters:

SMOOTH: smooth generated mask before optimizing.
SHOW_MASKS: to visualize the generated masks during optimization.
SIGMA: to control the fall off in the radial basis function when generating the masks. Play with its value to get different results; generally, 0.25 and 0.3 work well in most cases.
PALETTE_SIZE: number of colors in each palette.
ADD_BLACK_WHITE: to append black and white colors to the final palette before optimizing.
STYLE_LOSS_WEIGHT: weight of style loss
CONTENT_LOSS_WEIGHT: weight of content loss.
COLOR_DISTANCE: similarity metric when computing the mask. Options include: 'chroma_L2' (L2 on chroma space) or 'L2' (L2 on RGB space).
STYLE_FEATURE_DISTANCE: similarity metric for style loss. Options include: 'L2' or 'COSINE' (for cosine similarity).
CONTENT_FEATURE_DISTANCE: = similarity metric for content loss. Options include: 'L2' or 'COSINE' (for cosine similarity).
OPTIMIZER: optimization algorithm. Options include: 'LBFGS', 'Adam', 'Adagrad'.

MIT License

Related Research Projects

HistoGAN: A method to control colors of GAN and real images. Image auto recoloring is one of its applications.
Image Recoloring Based on Object Color Distributions: A method to perform automatic image recoloring based on the distribution of colors associated with objects present in an image.

jacobswan1 / color-aware-style-transfer