Wagtail Images De-duplicator

wagtail-images-deduplicator is a Wagtail app to detect duplicate images in the admin. It's built with imagehash.

Requirements

Wagtail Images De-duplicator works with wagtail>=3.0.

Installation

Use pip to install this package:

pip install wagtail-images-deduplicator

Configuration

Add wagtail_images_deduplicator to your INSTALLED_APPS in your project's settings.
Add the DuplicateFindingMixin to your custom image model. An example of doing it is shown below:

from wagtail.images.models import Image, AbstractImage, AbstractRendition

from wagtail_images_deduplicator.models import DuplicateFindingMixin


class CustomImage(DuplicateFindingMixin, AbstractImage):
    admin_form_fields = Image.admin_form_fields


class CustomRendition(AbstractRendition):
    image = models.ForeignKey(
        CustomImage, on_delete=models.CASCADE, related_name="renditions"
    )

    class Meta:
        unique_together = (("image", "filter_spec", "focal_point_key"),)

If you choose to add the mixin and have existing image data, you will need to call save() on all existing instances to fill in the new hash value:

from wagtail.images import get_image_model

for image in get_image_model().objects.all():
    image.save()

Settings

`WAGTAILIMAGESDEDUPLICATOR_HASH_FUNC`

This setting determines the hash function to use.

Hash function	Reference	Setting name
Average hashing	http://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html	`average_hash`
Perceptual hashing	http://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html	`phash` (default)
Difference hashing	http://www.hackerfactor.com/blog/index.php?/archives/529-Kind-of-Like-That.html	`dhash` or `dhash_vertical`
Wavelet hashing	https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/	`whash`
HSV color hashing		`colorhash`
Crop-resistant hashing	https://ieeexplore.ieee.org/document/6980335	`crop_resistant_hash`

`WAGTAILIMAGESDEDUPLICATOR_MAX_DISTANCE_THRESOLD`

This setting determines the maximum distance between 2 images to consider them as duplicates.
The default value is 5.

To help you assess how these different algorithms behave and to learn more about hash distances, check out the examples section of the imagehash library's README.

Tijani-Dia / wagtail-images-deduplicator