plaip / caire

Content aware image resize library

Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.

How does it work

• An energy map (edge detection) is generated from the provided image.
• The algorithm tries to find the least important parts of the image taking into account the lowest energy values.
• Using a dynamic programming approach the algorithm will generate individual seams accrossing the image from top to down, or from left to right (depending on the horizontal or vertical resizing) and will allocate for each seam a custom value, the least important pixels having the lowest energy cost and the most important ones having the highest cost.
• Traverse the image from the second row to the last row and compute the cumulative minimum energy for all possible connected seams for each entry.
• The minimum energy level is calculated by summing up the current pixel with the lowest value of the neighboring pixels from the previous row.
• Traverse the image from top to bottom and compute the minimum energy level. For each pixel in a row we compute the energy of the current pixel plus the energy of one of the three possible pixels above it.
• Find the lowest cost seam from the energy matrix starting from the last row and remove it.
• Repeat the process.

The process illustrated:

Original image	Energy map	Seams applied

Features

Key features which differentiates from the other existing open source solutions:

• Customizable command line support
• Support for both shrinking or enlarging the image
• Resize image both vertically and horizontally
• Can resize all the images from a directory
• Does not require any third party library
• Use of sobel threshold for fine tuning
• Use of blur filter for increased edge detection
• Make the image square with a single command
• Support for proportional scaling
• Face detection to avoid face deformation

Update

The library now supports face detection via https://gocv.io/, which is a Go binding for OpenCV 3. This means you need to install OpenCV 3 prior using this feature, otherwise check gocv documentation for the steps needed to install OpenCV 3.4.

In a future release i'm planning to implement my own face detection library to minimize the dependency tree. Until then if you whish to avoid extra installation you can use the binary files from the releases folder, or brew formulae if you are a MacOS user, but in this case you will miss the face detection feature.

Notice: gocv should be installed separately, otherwise you will get some OpenCV related errors! This is the reason why it was not included as dependency.

Just to illustrate the differences between face detection applied and without face detection, it's clearly visible that with face detection activated the algorithm will avoid to crop pixels inside faces.

Original image	With face detection	Without face detection

Sample image source

Install

First, install Go, set your GOPATH, and make sure $GOPATH/bin is on your PATH.

$ export GOPATH="$HOME/go"
$ export PATH="$PATH:$GOPATH/bin"

Next download the project and build the binary file.

$ go get -u -f github.com/esimov/caire/cmd/caire
$ go install

MacOS (Brew) install

The library now can be installed via Homebrew. The only thing you need is to run the commands below.

$ brew tap esimov/caire
$ brew install caire

Usage

$ caire -in input.jpg -out output.jpg

Supported commands:

$ caire --help

The following flags are supported:

Flag	Default	Description
in	n/a	Input file
out	n/a	Output file
width	n/a	New width
height	n/a	New height
perc	false	Reduce image by percentage
square	false	Reduce image to square dimensions
scale	false	Proportional scaling
blur	1	Blur radius
sobel	10	Sobel filter threshold
debug	false	Use debugger
face	false	Use face detection
xml	string	XML Classifier

In case you wish to scale down the image by a specific percentage, it can be used the -perc boolean flag. For example to reduce the image dimension by 20% both horizontally and vertically you can use the following command:

caire -in input/source.jpg -out ./out.jpg -perc=1 -width=20 -height=20 -debug=false

Also the library supports the -square option. When this option is used the image will be resized to a squre, based on the shortest edge.

The -scale option will resize the image proportionally. First the image is scaled down preserving the image aspect ratio, then the seam carving algorithm is applied only to the remaining points. Ex. : given an image of dimensions 2048x1536 if we want to resize to the 1024x500, the tool first rescale the image to 1024x768, then will remove only the remaining 268px. Using this option will drastically reduce the processing time.

The CLI command can process all the images from a specific directory too.

$ caire -in ./input-directory -out ./output-directory

Sample images

Shrunk images

Original	Shrunk

Enlarged images

Original	Extended

Useful resources

• https://en.wikipedia.org/wiki/Seam_carving
• https://inst.eecs.berkeley.edu/~cs194-26/fa16/hw/proj4-seamcarving/imret.pdf
• http://pages.cs.wisc.edu/~moayad/cs766/download_files/alnammi_cs_766_final_report.pdf
• https://stacks.stanford.edu/file/druid:my512gb2187/Zargham_Nassirpour_Content_aware_image_resizing.pdf

License

This project is under the MIT License. See the LICENSE file for the full license text.