duplicate-files

There are 8 repositories under duplicate-files topic.

• diskover-community

  diskoverdata / diskover-community

  Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch

  pythonelasticsearchcrawlerfilesystem-visualizationfilesystem-analysisfilesystem-indexerdisk-spacedisk-usagestorage-analyticsstoragefilesystemfile-indexingduplicatefilefindermetadataduplicate-filesfile-taggingdisk-space-analyzerweb-applicationphpfile-manager
  Language:PHP 1396

• m-manu / go-find-duplicates

  Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.

  command-line-toolduplicate-filesfile-scannergogolanggolang-applicationmediautilities
  Language:Go 258
• deduplicator

  sreedevk / deduplicator

  Filter, Sort & Delete Duplicate Files Recursively

  deduplicationduplicate-detectionduplicate-filesduplicatefilefinderfilesystemrust
  Language:Rust 254

• jvirkki / dupd

  CLI utility to find duplicate files

  duplicatesduplicatefilefindercduplicate-filesdeduplicationfdupes
  Language:C 110

• elmindreda / duff

  Command-line utility for finding duplicate files

  cduplicate-filesunix
  Language:C 88

• deplicate / deplicate

  Advanced Duplicate File Finder for Python

  duplicate-filesduplicatefilefinderduplicate-detectionduplicatesduplicateduplicates-removedscanningfinderpypipythonpurge-duplicate-filesmulti-filteringunixwindowsmacosxdeplicateduplication-finder
  Language:Python 76
• cbird

  scrubbbbs / cbird

  Command-line program for managing a media collection, with focus on Content-Based Image Retrieval (Computer Vision) methods for finding duplicates.

  command-line-interfacecontent-based-image-retrievalcomputer-visionduplicate-detectionduplicate-filesduplicatessimilarity-searchopencvqt6ffmpeg
  Language:C++ 71
• dude

  PJDude / dude

  Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.

  duplicate-detectionduplicate-filesduplicatesduplicates-removalpythontkinterdeduplicationsha1utilityutility-applicationpython3guigui-applicationthreadseasyeasy-to-useeasyuiduplicatecli
  Language:Python 53

• VISWESWARAN1998 / CyberGod-KSGMPRH

  An open-source antivirus for windows

  antivirusanti-malwareduplicate-filesrecoverydatabase
  Language:C 44

• fffaraz / QtHashSum

  File Checksum Integrity Verifier & Duplicate File Finder written in C++ Qt

  qthashc-plus-plussha1sha3md5cryptographic-hashsha1summd5sumchecksumduplicate-detectionchecksum-integrity-verifierduplicate-fileshashingfilesystem-indexerduplicatefilefinderintegrity
  Language:C++ 40

• neatdecisions / detwinner

  Detwinner - duplicate file finder for the Linux desktop

  linux-desktopduplicate-filesexact-duplicatessimilar-images
  Language:C++ 32

• Hopfengetraenk / DFHL

  Duplicate File Hard Linker

  hardlinkntfsduplicate-filescommand-line-tool
  Language:C++ 29

• caluml / finddups

  Find duplicate files on your computer

  javaduplicate-filesduplicatescommand-linecommand-line-toolsimple
  Language:Java 21

• github-userx / Awesome-Duplication-Finders

  Apps to find duplicate files including same/similar images & videos (with computer vision/AI)

  deduplicationduplicate-detectionduplicate-filesduplicate-imagesduplicate-video-finderduplicate-videosduplicatefilefinderduplicatesduplicationduplication-finder
  20
• Data-Duplication-Removal-using-Machine-Learning

  Vatshayan / Data-Duplication-Removal-using-Machine-Learning

  Final Year Project as Deletion of Duplicated data using Machine learning project with source code and Report.

  final-year-projectcse-projectdata-duplicationchecksumchecksumsduplicate-filespython-projectminor-projectmajor-projectcollege-projectmachine-learningmachine-learning-projectsfinal-projectsemester-projectdata-duplication-removalproject-developerprojects-developerproject-maker
  Language:Jupyter Notebook 20

• mcguffin / wp-unique-media

  WordPress plugin to prevent duplicate files from being uploaded.

  wordpress-pluginmedia-libraryduplicate-files
  Language:PHP 17

• rustomax / ndf

  Duplicate file finder written in Nim

  duplicate-detectionduplicate-filesduplicatesfilesystemndfnimutility
  Language:Nim 17

• JiuLing-zhang / FindDuplicateFiles

  .NET WPF 写的重复文件查找工具，解放你的硬盘空间，红盘用户的福音。

  duplicate-fileduplicate-filesduplicatefileduplicatefilefinder
  Language:C# 15

• raspi / duplikaatti

  Remove duplicate files.

  gogolangduplicate-filesduplicatesfiles
  Language:Go 15

• DeaDSouL / dugu

  Find, remove and avoid duplicates with dugu: The Duplicates Guru

  pythonduplicate-detectionduplicatesduplicatefilefinderduplicate-filesduguduplicates-gurudeduplication
  Language:Python 13

• vishal-wadhwa / Duplicate-File-Remover

  Command line utility to remove exact duplicate files.

  duplicate-filesduplicatefilefinderduplicate-imagesduplicates-removedduplicate-detectioncpp
  Language:C++ 13

• NicolasBizzozzero / dupe_eraser

  A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

  duplicatesclifile-managementduplicate-filesduplicate-detection
  Language:Python 12

• sameera-madushan / Findm

  Findm is a python script to find duplicate file copies in a given directory.

  duplicate-detectionduplicate-filesfile-hashingpythonduplicatefilefinder
  Language:Python 11

• thushan / smash

  Smash through to find duplicate files super fast by slicing files intelligently!

  clicli-toolduplicate-filesfreebsdgolinuxmacoswindows
  Language:Go 10

• MK-Ware / pyCleaner

  Securely wipe files or folders and clean duplicated files

  secure-wipingwipe-filesmd5hashingduplicate-filesduplicatefilefinderduplicate-detectionduplicationcleanercleaningpython3tkinter-graphic-interfaceremove-duplicatesshreddershredding-files
  Language:Python 9
• file-duplicate-checker

  Gartham / file-duplicate-checker

  Program to scan and search for file duplicates. (~300MB/s)

  duplicate-detectionduplicate-filesduplicate-imagesfile-duplicatorfile-managerfilemanagerfilesfileserverfilestoragecompress-imagescompressionduplicatefilefinderduplicates-removed
  Language:Java 8

• raspi / samanlainen

  Delete duplicate files

  duplicate-detectionduplicate-filesduplicatesfilesrust
  Language:Rust 6

• elcorto / findsame

  Find duplicate files and directories based on file hashes.

  merkletreemultithreadingfile-hashingduplicate-filesduplicate-detectionmultiprocessingpythonduplicatefilefinder
  Language:Python 5

• guofei9987 / fuzzy-hash

  fuzzy hash in python (from ssdeep)

  duplicate-detectionduplicate-filesfuzzy-hashhash
  Language:C 5

• jeff-99 / hashdex

  A file indexer based on content hashes to quickly find duplicate files on your system.

  fileindexinghashduplicatefilefinderduplicate-files
  Language:Python 5

• maanavshah / duplicate-file-finder

  A Java application with GUI(Swing) to find all the duplicate files in a given directory and all its sub-directories, using the SHA-512 hash function

  duplicate-filesjavaswing-gui
  Language:Java 5

• nogibjj / pyli

  Deduper

  litenduplicate-filesclideduplication-commandstdout
  Language:Python 5

• s1s1ty / Dropbox-Duplicate-Remover

  Remove duplicate files from dropbox

  dropboxduplicate-filespython
  Language:Python 5

• cobanov / easy-duplicate

  Compares the files in a folder with md5 checksums and deletes duplicate files or moves them to the desired folder.

  duplicateduplicate-detectionduplicate-filesduplicatespython
  Language:Python 4

• Jim-JMCD / Duplicate-File-Finder

  A bash script compares files using sha256. Input can be one or more directories with optional filters of maximum files size and parts of file names (e.g. ‘.mp4’ ‘IMG’ ) . Output is multiple CSV (spreadsheet) reports that can be used to move or delete duplicates that are found. Can be used on Windows WSL2, MSYS2 and Gitbash.

  duplicate-file-finderduplicate-filesduplicatefilefinderwsl2linuxduplicate-file-locater
  Language:Shell 4

• nagracks / py_duplicates

  Find duplicates

  duplicate-filesclipython-scriptscript
  Language:Python 4