duplicates

There are 12 repositories under duplicates topic.

• czkawka

  qarmin / czkawka

  Multi functional app to find duplicates, empty folders, similar images etc.

  cleanerduplicatesgtk-rsmultiplatformrustsimilar-imagessimilar-musicsimilar-videos
  Language:Rust 27000

• kucherenko / jscpd

  Copy/paste detector for programming source code.

  clones-detectioncode-qualitycopy-pastecpddetect-duplicationsdetectorduplicatesduplicationsquality
  Language:TypeScript 5073

• sahib / rmlint

  Extremely fast tool to remove duplicates and other lint from your filesystem

  cdeduplicationduplicatesfdupesfilesystemlintpython
  Language:C 2209

• scinos / yarn-deduplicate

  Deduplication tool for yarn.lock files

  yarnduplicatesdedupeduplicated-packagesyarn-locklock-file
  Language:TypeScript 1391

• kristiankoskimaki / vidupe

  Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

  videosduplicatesduplicate-detectionduplicate-videos
  Language:C++ 186

• F483 / dejavu

  Quickly detect already witnessed data.

  golanggohistorymemorydeduplicationduplicate-valuesduplicatescommand-linecommand-line-toolprobabilistic
  Language:Go 156
• dude

  PJDude / dude

  Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.

  clideduplicationduplicateduplicate-detectionduplicate-filesduplicatesduplicates-removaleasyeasy-to-useeasyuiguigui-applicationpythonpython3sha1threadstkinterutilityutility-application
  Language:Python 156

• Canop / backdown

  A deduplicator

  rustduplicates
  Language:Rust 141
• cbird

  scrubbbbs / cbird

  Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.

  command-line-interfacecomputer-visioncontent-based-image-retrievalduplicate-detectionduplicate-filesduplicatesffmpegopencvqt6similarity-search
  Language:C++ 140

• kouhin / redux-dataloader

  Loads async data for Redux apps focusing on preventing duplicated requests and dealing with async dependencies.

  reduxreactthunkasyncactionduplicates
  Language:JavaScript 139

• matteodelabre / mongoose-beautiful-unique-validation

  Plugin for Mongoose that turns duplicate errors into regular Mongoose validation errors

  mongooseduplicatesschemaerrorsvalidationplugin
  Language:JavaScript 118

• jvirkki / dupd

  CLI utility to find duplicate files

  duplicatesduplicatefilefindercduplicate-filesdeduplicationfdupes
  Language:C 116
• removedupes

  eyalroz / removedupes

  Remove Duplicate Messages

  thunderbirdmozillamail-clientemailduplicatesduplicate-detectionmail-foldersemail-parsingproductivitycleanupcleanerthunderbird-extensionthunderbird-addon
  Language:JavaScript 104

• cloud-py-api / mediadc

  Nextcloud Media Duplicate Collector application

  collectorduplicate-detectionduplicatesmediamediadcnextcloudnextcloud-appsnextcloud-vue-appopen-sourcephppythonpython3single-page-appvue
  Language:PHP 102

• microsoft / near-duplicate-code-detector

  A simple tool for detecting near-duplicate source code

  duplicatesdetect-duplicates
  Language:C# 100
• Katalog

  StephaneCouturier / Katalog

  Katalog is an application to manage catalogs of disks and files to search and get statistics.

  catalogingcataloguedifferencesduplicatesexternal-contentfilefile-managerfilemanagerfilesystemkatalogstoragefile-statisticsbackupindexersearchexternal-drives
  Language:C++ 98
• lastfm-smart-deduper

  shevchenkoartem / lastfm-smart-deduper

  JS script that allows you to remove duplicates from your Last.fm scrobbles library.

  lastfmlast-fmduplicatesscrobblingdeduplicateremove-duplicates
  Language:JavaScript 92

• vuolter / deplicate

  Advanced Duplicate File Finder for Python

  duplicate-filesduplicatefilefinderduplicate-detectionduplicatesduplicateduplicates-removedscanningfinderpypipythonpurge-duplicate-filesmulti-filteringunixwindowsmacosxdeplicateduplication-finder
  Language:Python 82

• github-userx / Awesome-Duplication-Finders

  Apps to find duplicate files including same/similar images & videos (with computer vision/AI)

  deduplicationduplicate-detectionduplicate-filesduplicate-imagesduplicate-video-finderduplicate-videosduplicatefilefinderduplicatesduplicationduplication-finder
  73
• bitwarden_find_duplicates

  eliasfloreteng / bitwarden_find_duplicates

  Find duplicate logins based on domain, from Bitwarden export. Open source for your safety.

  bitwardenduplicates
  Language:HTML 63

• twpayne / find-duplicates

  Find duplicate files quickly.

  duplicate-detectionduplicate-filesduplicatesfind
  Language:Go 61

• src-d / gemini

  Advanced similarity and duplicate source code at scale.

  duplicate-detectionsparkduplicateshashsource-code-analysis
  Language:Scala 56

• deric / es-dedupe

  Tool for removing duplicate documents from Elasticsearch

  elasticsearchduplicatesduplicity
  Language:Python 54

• src-d / apollo

  Advanced similarity and duplicate source code proof of concept for our research efforts.

  duplicatessimilaritysource-codeduplicate-detectionsimilarity-searchpython
  Language:Python 52

• LyonSyonII / akin

  Rust crate for writing repetitive code easier and faster.

  rustrust-craterepetitionrepeatduplicateduplicatessimilarsimilarswritecodesimpler
  Language:Rust 45

• Bartozzz / potential-duplicates-bot

  A configurable GitHub App which checks for potential issue duplicates using Damerau–Levenshtein distance algorithm.

  githubprobotprobot-appprobot-pluginissuesduplicatesbot
  Language:JavaScript 31

• guicamest / GDuplicate-Finder

  GDuplicate Finder - A Groovy way to find duplicates among your computer and network shares!

  duplicateduplicatesfinders3network-sharesshgparsgroovy
  Language:Groovy 30

• gacarrillor / AppendFeaturesToLayer

  QGIS Processing plugin to add an algorithm for upserting features from a source vector layer to an existing target vector layer.

  qgisqgis3-pluginqgis3qgis-processingqgis-processing-providerappendetlloadload-datacopyupdatecopy-pasteduplicatesupsertexport
  Language:Python 26

• Navid2zp / dups

  A CLI tool to find/remove duplicate files supporting multi-core and different algorithms (MD5, SHA256, and XXHash).

  duplicatesgogolangmd5sha256xxhash
  Language:Go 26

• arikw / outlook-duplicated-items-remover

  A VBA script that finds and moves duplicated items in selected outlook folders

  outlookduplicate-detectionduplicatesvbavba-scriptvba-snippets
  Language:VBA 22

• rasheedsulayman / DuplicateContactsRemover

  📒 A simple app to optimize your address book and remove duplicate contacts.

  kotlinmaterial-designarchitecture-componentsandroidxcontacts-managermvvmduplicatesdependency-injection
  Language:Kotlin 22

• caluml / finddups

  Find duplicate files on your computer

  javaduplicate-filesduplicatescommand-linecommand-line-toolsimple
  Language:Java 21

• rustomax / ndf

  Duplicate file finder written in Nim

  duplicate-filesndfnimduplicatesduplicate-detectionfilesystemutility
  Language:Nim 20

• mkearney / funique

  ⌚️ A faster unique() function

  data-framedata-wranglingdate-timeduplicatesmkearney-r-packageposixposixctrr-packagerstatsunique
  Language:R 19

• vladsmirnov / url-rewrites

  Magento 1.x module to target the URL Rewrite issue

  magentoduplicatesmodmancomposerreindex
  Language:PHP 19

• cemahseri / Duplica

  A very fast duplicate file finder.

  deduplicationduplicateduplicate-detectionduplicate-file-finderduplicate-filesduplicatefilefinderduplicatefilesduplicatesduplicatedetection
  Language:C# 18