duplicate-detection
There are 15 repositories under duplicate-detection topic.
windirstat / windirstat
WinDirStat is a disk usage statistics viewer and cleanup tool for Microsoft Windows
Language:C++ 2160nomic-ai / nomic
Interact, analyze and structure massive text, image, embedding, audio and video datasets
Language:Python 1851- duplicut
nil0x42 / duplicut
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
Language:C++ 960 - zoplicate
ChenglongMa / zoplicate
A plugin that does one thing only: Detect and manage duplicate items in Zotero.
Language:TypeScript 703 - deduplicator
sreedevk / deduplicator
Filter, Sort & Delete Duplicate Files Recursively
Language:Rust 352 - videohash
akamhy / videohash
Near Duplicate Video Detection (Perceptual Video Hashing) - Get a 64-bit comparable hash-value for any video.
Language:Python 336 RazgrizHsu / immich-mediakit
An extension toolkit for Immich enabling advanced management capabilities through AI-powered similarity detection
Language:Python 298- depp
CryogenicPlanet / depp
⚡ Check your npm modules for unused and duplicate dependencies fast
Language:Go 267 - Language:Java 236
- SIFTImageSimilarity
adumrewal / SIFTImageSimilarity
Interactive code for image similarity using SIFT algorithm
Language:Python 225 kristiankoskimaki / vidupe
Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:
Language:C++ 185apastel / ytmusic-deleter
Easily delete your YouTube Music library (and manage playlists)
Language:Python 175markusressel / py-image-dedup
CLI utility to find near duplicate images and remove all but the best copy.
Language:Python 166umbertogriffo / fast-near-duplicate-image-search
Fast Near-Duplicate Image Search and Delete using pHash, t-SNE and KDTree.
Language:Python 163unmade / audiomatch
Find similar audio files easily
Language:Python 158- dude
PJDude / dude
Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.
Language:Python 156 - cbird
scrubbbbs / cbird
Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.
Language:C++ 137 itwillwork / ostap
CLI tool that fast checks if your bundle contains multiple versions of the same package, only by looking in package.json.
Language:JavaScript 116- Language:C++ 115
- removedupes
eyalroz / removedupes
Remove Duplicate Messages
Language:JavaScript 104 cloud-py-api / mediadc
Nextcloud Media Duplicate Collector application
Language:PHP 100Chinmay26 / Near-Duplicate-Video-Detection
Detecting near-duplicate videos by aggregating features from intermediate CNN layers
Language:Jupyter Notebook 99- pyJedAI
AI-team-UoA / pyJedAI
An open-source library that leverages Python’s data science ecosystem to build powerful end-to-end Entity Resolution workflows.
Language:Python 84 - Language:Python 81
github-userx / Awesome-Duplication-Finders
Apps to find duplicate files including same/similar images & videos (with computer vision/AI)
marius-sucan / Quick-Picto-Viewer
A uniquely crafted image viewer and editor with options to organize files, and maintain large lists of image files for slideshows, dupes detection or other purposes.
Language:AutoHotkey 64- Language:Rust 63
twpayne / find-duplicates
Find duplicate files quickly.
Language:Go 61- Language:Scala 56
src-d / apollo
Advanced similarity and duplicate source code proof of concept for our research efforts.
Language:Python 52- Language:C++ 48
- Language:JavaScript 47
sed-inf-u-szeged / OpenStaticAnalyzer
OpenStaticAnalyzer is a source code analyzer tool, which can perform deep static analysis of the source code of complex systems.
Language:C++ 47- snowman
HPI-Information-Systems / snowman
Welcome to Snowman App – a Data Matching Benchmark Platform.
Language:TypeScript 38 - Language:PHP 37
