duplicates

There are 8 repositories under duplicates topic.

• czkawka

  qarmin / czkawka

  Multi functional app to find duplicates, empty folders, similar images etc.

  cleanerduplicatesgtk-rsmultiplatformrustsimilar-imagessimilar-musicsimilar-videos
  Language:Rust 17681

• kucherenko / jscpd

  Copy/paste detector for programming source code.

  detectorqualitycopy-pasteduplicatescpdcode-qualityduplicationsdetect-duplicationsclones-detection
  Language:TypeScript 4542

• sahib / rmlint

  Extremely fast tool to remove duplicates and other lint from your filesystem

  cdeduplicationduplicatesfdupesfilesystemlintpython
  Language:C 1787

• scinos / yarn-deduplicate

  Deduplication tool for yarn.lock files

  yarnduplicatesdedupeduplicated-packagesyarn-locklock-file
  Language:TypeScript 1375

• sean-public / python-hashes

  Interesting (non-cryptographic) hashes implemented in pure Python.

  pythonpython3hashhash-functionshashes-implementedbloom-filternilsimsageohashessimhashduplicates
  Language:Python 238

• F483 / dejavu

  Quickly detect already witnessed data.

  golanggohistorymemorydeduplicationduplicate-valuesduplicatescommand-linecommand-line-toolprobabilistic
  Language:Go 158

• kouhin / redux-dataloader

  Loads async data for Redux apps focusing on preventing duplicated requests and dealing with async dependencies.

  reduxreactthunkasyncactionduplicates
  Language:JavaScript 140

• kristiankoskimaki / vidupe

  Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

  videosduplicatesduplicate-detectionduplicate-videos
  Language:C++ 135

• matteodelabre / mongoose-beautiful-unique-validation

  Plugin for Mongoose that turns duplicate errors into regular Mongoose validation errors

  duplicateserrorsmongoosepluginschemavalidation
  Language:JavaScript 117

• jvirkki / dupd

  CLI utility to find duplicate files

  duplicatesduplicatefilefindercduplicate-filesdeduplicationfdupes
  Language:C 109

• Canop / backdown

  A deduplicator

  duplicatesrust
  Language:Rust 96

• cloud-py-api / mediadc

  Nextcloud Media Duplicate Collector application

  nextcloudnextcloud-appsmediaduplicatesduplicate-detectioncollectormediadcopen-sourcesingle-page-appphpvuenextcloud-vue-apppythonpython3
  Language:PHP 93

• microsoft / near-duplicate-code-detector

  A simple tool for detecting near-duplicate source code

  duplicatesdetect-duplicates
  Language:C# 89
• removedupes

  eyalroz / removedupes

  Remove Duplicate Messages

  thunderbirdmozillamail-clientemailduplicatesduplicate-detectionmail-foldersemail-parsingproductivitycleanupcleanerthunderbird-extensionthunderbird-addon
  Language:JavaScript 79

• deplicate / deplicate

  Advanced Duplicate File Finder for Python

  duplicate-filesduplicatefilefinderduplicate-detectionduplicatesduplicateduplicates-removedscanningfinderpypipythonpurge-duplicate-filesmulti-filteringunixwindowsmacosxdeplicateduplication-finder
  Language:Python 76
• cbird

  scrubbbbs / cbird

  Command-line program for managing a media collection, with focus on Content-Based Image Retrieval (Computer Vision) methods for finding duplicates.

  command-line-interfacecomputer-visioncontent-based-image-retrievalduplicate-detectionduplicate-filesduplicatesffmpegopencvqt6similarity-search
  Language:C++ 75
• lastfm-smart-deduper

  shevchenkoartem / lastfm-smart-deduper

  JS script that allows you to remove duplicates from your Last.fm scrobbles library.

  lastfmlast-fmduplicatesscrobblingdeduplicateremove-duplicates
  Language:JavaScript 71
• Katalog

  StephaneCouturier / Katalog

  Katalog is an application to manage catalogs of disks and files to search and get statistics.

  catalogingcataloguedifferencesduplicatesexternal-contentfilefile-managerfilemanagerfilesystemkatalogstoragefile-statisticsbackupindexersearchexternal-drives
  Language:C++ 58
• dude

  PJDude / dude

  Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.

  duplicate-detectionduplicate-filesduplicatesduplicates-removalpythontkinterdeduplicationsha1utilityutility-applicationpython3guigui-applicationthreadseasyeasy-to-useeasyuiduplicatecli
  Language:Python 55

• deric / es-dedupe

  Tool for removing duplicate documents from Elasticsearch

  duplicatesduplicityelasticsearch
  Language:Python 54

• src-d / gemini

  Advanced similarity and duplicate source code at scale.

  duplicate-detectionsparkduplicateshashsource-code-analysis
  Language:Scala 54

• src-d / apollo

  Advanced similarity and duplicate source code proof of concept for our research efforts.

  duplicatessimilaritysource-codeduplicate-detectionsimilarity-searchpython
  Language:Python 52

• LyonSyonII / akin

  Rust crate for writing repetitive code easier and faster.

  rustrust-craterepetitionrepeatduplicateduplicatessimilarsimilarswritecodesimpler
  Language:Rust 42

• Bartozzz / potential-duplicates-bot

  A configurable GitHub App which checks for potential issue duplicates using Damerau–Levenshtein distance algorithm.

  githubprobotprobot-appprobot-pluginissuesduplicatesbot
  Language:JavaScript 31

• guicamest / GDuplicate-Finder

  GDuplicate Finder - A Groovy way to find duplicates among your computer and network shares!

  duplicateduplicatesfindergparsgroovynetwork-shares3ssh
  Language:Groovy 30

• Navid2zp / dups

  A CLI tool to find/remove duplicate files supporting multi-core and different algorithms (MD5, SHA256, and XXHash).

  duplicatesgogolangmd5sha256xxhash
  Language:Go 23

• gacarrillor / AppendFeaturesToLayer

  Processing plugin-based provider for QGIS 3 that adds an algorithm for appending/updating features from a source vector layer to an existing target vector layer.

  qgisqgis3-pluginqgis3qgis-processingqgis-processing-providerappendetlloadload-datacopyupdatecopy-pasteduplicatesupsertexport
  Language:Python 22

• caluml / finddups

  Find duplicate files on your computer

  javaduplicate-filesduplicatescommand-linecommand-line-toolsimple
  Language:Java 21

• github-userx / Awesome-Duplication-Finders

  Apps to find duplicate files including same/similar images & videos (with computer vision/AI)

  deduplicationduplicate-detectionduplicate-filesduplicate-imagesduplicate-video-finderduplicate-videosduplicatefilefinderduplicatesduplicationduplication-finder
  21

• rasheedsulayman / DuplicateContactsRemover

  📒 A simple app to optimize your address book and remove duplicate contacts.

  kotlinmaterial-designarchitecture-componentsandroidxcontacts-managermvvmduplicatesdependency-injection
  Language:Kotlin 21

• mkearney / funique

  ⌚️ A faster unique() function

  rrstatsuniqueduplicatesr-packagemkearney-r-packagedata-wranglingdata-framedate-timeposixctposix
  Language:R 19

• vladsmirnov / url-rewrites

  Magento 1.x module to target the URL Rewrite issue

  magentoduplicatesmodmancomposerreindex
  Language:PHP 19

• arikw / outlook-duplicated-items-remover

  A VBA script that finds and moves duplicated items in selected outlook folders

  duplicate-detectionduplicatesoutlookvbavba-scriptvba-snippets
  Language:VBA 18

• kevinpollet / pocket-deduper

  Remove duplicates from your Pocket list.

  gogolangpocketdedupeduplicatestoolcli
  Language:Go 18

• lmammino / indexed-string-variation

  Experimental JavaScript module to generate all possible variations of strings over an alphabet using an n-ary virtual tree

  algorithmalphabetcharactersduplicatesjavascriptjavascript-librarylibrarynodenodejsstringstringstreevariationsvirtual
  Language:JavaScript 18
• bitwarden_find_duplicates

  elias123tre / bitwarden_find_duplicates

  Find duplicate logins based on domain, from bitwarden export. Open source for your safety.

  bitwardenduplicates
  Language:HTML 17