duplicates

There are 10 repositories under duplicates topic.

• czkawka

  qarmin / czkawka

  Multi functional app to find duplicates, empty folders, similar images etc.

  cleanerduplicatesgtk-rsmultiplatformrustsimilar-imagessimilar-musicsimilar-videos
  Language:Rust 19433

• kucherenko / jscpd

  Copy/paste detector for programming source code.

  detectorqualitycopy-pasteduplicatescpdcode-qualityduplicationsdetect-duplicationsclones-detection
  Language:TypeScript 4664

• sahib / rmlint

  Extremely fast tool to remove duplicates and other lint from your filesystem

  cpythonfilesystemlintduplicatesfdupesdeduplication
  Language:C 1904

• scinos / yarn-deduplicate

  Deduplication tool for yarn.lock files

  dedupeduplicated-packagesduplicateslock-fileyarnyarn-lock
  Language:TypeScript 1382

• sean-public / python-hashes

  Interesting (non-cryptographic) hashes implemented in pure Python.

  pythonpython3hashhash-functionshashes-implementedbloom-filternilsimsageohashessimhashduplicates
  Language:Python 240

• F483 / dejavu

  Quickly detect already witnessed data.

  golanggohistorymemorydeduplicationduplicate-valuesduplicatescommand-linecommand-line-toolprobabilistic
  Language:Go 157

• kristiankoskimaki / vidupe

  Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

  videosduplicatesduplicate-detectionduplicate-videos
  Language:C++ 146

• kouhin / redux-dataloader

  Loads async data for Redux apps focusing on preventing duplicated requests and dealing with async dependencies.

  reduxreactthunkasyncactionduplicates
  Language:JavaScript 139

• matteodelabre / mongoose-beautiful-unique-validation

  Plugin for Mongoose that turns duplicate errors into regular Mongoose validation errors

  mongooseduplicatesschemaerrorsvalidationplugin
  Language:JavaScript 117

• Canop / backdown

  A deduplicator

  duplicatesrust
  Language:Rust 116

• jvirkki / dupd

  CLI utility to find duplicate files

  cdeduplicationduplicate-filesduplicatefilefinderduplicatesfdupes
  Language:C 112
• dude

  PJDude / dude

  Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.

  clideduplicationduplicateduplicate-detectionduplicate-filesduplicatesduplicates-removaleasyeasy-to-useeasyuiguigui-applicationpythonpython3sha1threadstkinterutilityutility-application
  Language:Python 107

• cloud-py-api / mediadc

  Nextcloud Media Duplicate Collector application

  nextcloudnextcloud-appsmediaduplicatesduplicate-detectioncollectormediadcopen-sourcesingle-page-appphpvuenextcloud-vue-apppythonpython3
  Language:PHP 94

• microsoft / near-duplicate-code-detector

  A simple tool for detecting near-duplicate source code

  detect-duplicatesduplicates
  Language:C# 90
• cbird

  scrubbbbs / cbird

  Command-line program for managing a media collection, with focus on Content-Based Image Retrieval (Computer Vision) methods for finding duplicates.

  command-line-interfacecomputer-visioncontent-based-image-retrievalduplicate-detectionduplicate-filesduplicatesffmpegopencvqt6similarity-search
  Language:C++ 89
• removedupes

  eyalroz / removedupes

  Remove Duplicate Messages

  cleanercleanupduplicate-detectionduplicatesemailemail-parsingmail-clientmail-foldersmozillaproductivitythunderbirdthunderbird-addonthunderbird-extension
  Language:JavaScript 87

• deplicate / deplicate

  Advanced Duplicate File Finder for Python

  duplicate-filesduplicatefilefinderduplicate-detectionduplicatesduplicateduplicates-removedscanningfinderpypipythonpurge-duplicate-filesmulti-filteringunixwindowsmacosxdeplicateduplication-finder
  Language:Python 77
• lastfm-smart-deduper

  shevchenkoartem / lastfm-smart-deduper

  JS script that allows you to remove duplicates from your Last.fm scrobbles library.

  lastfmlast-fmduplicatesscrobblingdeduplicateremove-duplicates
  Language:JavaScript 75
• Katalog

  StephaneCouturier / Katalog

  Katalog is an application to manage catalogs of disks and files to search and get statistics.

  catalogingcataloguedifferencesduplicatesexternal-contentfilefile-managerfilemanagerfilesystemkatalogstoragefile-statisticsbackupindexersearchexternal-drives
  Language:C++ 62

• deric / es-dedupe

  Tool for removing duplicate documents from Elasticsearch

  elasticsearchduplicatesduplicity
  Language:Python 54

• src-d / gemini

  Advanced similarity and duplicate source code at scale.

  duplicate-detectionsparkduplicateshashsource-code-analysis
  Language:Scala 54

• src-d / apollo

  Advanced similarity and duplicate source code proof of concept for our research efforts.

  duplicatessimilaritysource-codeduplicate-detectionsimilarity-searchpython
  Language:Python 52

• LyonSyonII / akin

  Rust crate for writing repetitive code easier and faster.

  rustrust-craterepetitionrepeatduplicateduplicatessimilarsimilarswritecodesimpler
  Language:Rust 43

• github-userx / Awesome-Duplication-Finders

  Apps to find duplicate files including same/similar images & videos (with computer vision/AI)

  deduplicationduplicate-detectionduplicate-filesduplicate-imagesduplicate-video-finderduplicate-videosduplicatefilefinderduplicatesduplicationduplication-finder
  32

• Bartozzz / potential-duplicates-bot

  A configurable GitHub App which checks for potential issue duplicates using Damerau–Levenshtein distance algorithm.

  githubprobotprobot-appprobot-pluginissuesduplicatesbot
  Language:JavaScript 31

• guicamest / GDuplicate-Finder

  GDuplicate Finder - A Groovy way to find duplicates among your computer and network shares!

  duplicateduplicatesfinders3network-sharesshgparsgroovy
  Language:Groovy 30
• bitwarden_find_duplicates

  elias123tre / bitwarden_find_duplicates

  Find duplicate logins based on domain, from Bitwarden export. Open source for your safety.

  bitwardenduplicates
  Language:HTML 26

• Navid2zp / dups

  A CLI tool to find/remove duplicate files supporting multi-core and different algorithms (MD5, SHA256, and XXHash).

  golanggoxxhashduplicatesmd5sha256
  Language:Go 24

• gacarrillor / AppendFeaturesToLayer

  QGIS Processing plugin to add an algorithm for upserting features from a source vector layer to an existing target vector layer.

  appendcopycopy-pasteduplicatesetlexportloadload-dataqgisqgis-processingqgis-processing-providerqgis3qgis3-pluginupdateupsert
  Language:Python 23

• caluml / finddups

  Find duplicate files on your computer

  javaduplicate-filesduplicatescommand-linecommand-line-toolsimple
  Language:Java 21

• rasheedsulayman / DuplicateContactsRemover

  📒 A simple app to optimize your address book and remove duplicate contacts.

  kotlinmaterial-designarchitecture-componentsandroidxcontacts-managermvvmduplicatesdependency-injection
  Language:Kotlin 21

• rustomax / ndf

  Duplicate file finder written in Nim

  duplicate-filesndfnimduplicatesduplicate-detectionfilesystemutility
  Language:Nim 20

• arikw / outlook-duplicated-items-remover

  A VBA script that finds and moves duplicated items in selected outlook folders

  outlookduplicate-detectionduplicatesvbavba-scriptvba-snippets
  Language:VBA 19

• mkearney / funique

  ⌚️ A faster unique() function

  rrstatsuniqueduplicatesr-packagemkearney-r-packagedata-wranglingdata-framedate-timeposixctposix
  Language:R 19

• vladsmirnov / url-rewrites

  Magento 1.x module to target the URL Rewrite issue

  magentoduplicatesmodmancomposerreindex
  Language:PHP 19

• kevinpollet / pocket-deduper

  Remove duplicates from your Pocket list.

  gogolangpocketdedupeduplicatestoolcli
  Language:Go 18