distributed-file-system

There are 70 repositories under distributed-file-system topic.

• seaweedfs

  seaweedfs / seaweedfs

  SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

  blob-storagecloud-drivedistributed-file-systemdistributed-storagedistributed-systemserasure-codingfusehadoop-hdfshdfskubernetesobject-storageposixreplications3s3-storageseaweedfstiered-file-system
  Language:Go 27303
• ceph

  ceph / ceph

  Ceph is a distributed object, block, and file storage platform

  block-storagecloud-storagedistributed-file-systemdistributed-storageerasure-codingfusehdfshigh-performancehighly-availableiscsikubernetesnfsnvme-over-fabricsobject-storeposixreplications3smbsoftware-defined-storagestorage
  Language:C++ 15695

• deepseek-ai / 3FS

  A high-performance distributed file system designed to address the challenges of AI training and inference workloads.

  distributed-file-system
  Language:C++ 9444

• happyfish100 / fastdfs

  FastDFS is a high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balance. Wechat/Weixin public account (Chinese Language): fastdfs

  distributed-file-systemdistributed-file-storagestorage-servers
  Language:C 9204

• cubefs / cubefs

  cloud-native distributed storage

  ai-native-storagecloud-native-storagecloud-storagecncfdata-orchestrationdistributed-file-systemdistributed-storageerasure-codingfusehybrid-cloudkubernetesobject-storage
  Language:Go 5301

• sjqzhang / go-fastdfs

  go-fastdfs 是一个简单的分布式文件系统(私有云存储)，具有无中心、高性能，高可靠，免维护等优点，支持断点续传，分块上传，小文件合并，自动同步，自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

  storage-serversdistributed-file-storagedistributed-file-systemgodfsfile-systemgo-fastdfsfastdfsbreakpoint-resumesmall-file-mergetusfileserversyncrsyncgostoragecloudnativecloud-storagefilesystemossobject-storage-service
  Language:Go 4103

• leo-project / leofs

  The LeoFS Storage System

  leofserlangdistributed-storagedistributed-file-systems3-storages3nfs-servernfsdatalake
  Language:Erlang 1575

• wenweihu86 / raft-java

  Raft Java implementation which is simple and easy to understand.

  distributed-file-systemdistributed-storageraftraft-algorithmraft-consensusraft-consensus-algorithmraft-javaraft-protocol
  Language:Java 1229
• awesome-distributed-system-projects

  roma-glushko / awesome-distributed-system-projects

  🚀 List of distributed system projects for inspiration and learning to build distributed services from real world examples

  distributed-systemsdistributed-databasedistributed-file-systemdistributed-lockhighly-reliable-systemsgolangjavacpperlang
  964

• Hexilee / tifs

  A distributed POSIX filesystem based on TiKV, with partition tolerance and strict consistency.

  distributed-file-systemdistributed-systemsfilesystemfuseposix-compatibletifstikv
  Language:Rust 363

• hegongshan / File-System-Paper

  Must-read Papers for File System (FS)

  distributed-file-systemfile-systemlocal-file-systemcephkernel-file-systemuser-space-file-systemcrash-consistencyfragmentationmetadata-managementmany-core-scalabilitydata-distributionfault-tolerancehdfs
  309

• freakmaxi / kertish-dos

  Kertish-dos is a simple distributed object storage platform, implements object storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dos aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

  distributed-storagedistributed-file-systemdistributed-file-storagedfsdata-farmscalabledosobject-storages3-storagedistributed-object-storagedistributed-object-store
  Language:Go 109

• richardschneider / net-ipfs-engine

  IPFS Core API implementation in .Net

  ipfsdistributed-file-systempubsubpeer-to-peer
  Language:C# 109
• saunafs

  leil-io / saunafs

  SaunaFS is a free-and open source, distributed POSIX file system inspired by Google File System.

  archivingdistributed-file-systemdistributed-storagefault-tolerancehigh-availabilityhigh-performancehigh-performance-computingself-healingself-hosted
  Language:C++ 96

• pitchpoint-solutions / sfs

  The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

  distributed-file-systemdistributed-storagedistributed-systemselasticsearchjavaobject-storageopenstack-swiftrxjavavertx
  Language:Java 87

• IBM / IPFSfB

  InterPlanetary File System for Business (IPFSfB) is an enterprise blockchain storage network based on InterPlanetary File System.

  blockchainipfscryptographyp2pprotocoldistributed-file-systemnetworking
  Language:Go 79

• fangpin / miniDFS

  Simple version of Distributed File System on multiple threads

  minidfsdistributed-file-systemdistributedfile-system
  Language:C++ 59

• wenweihu86 / distkv

  Distributed KV Storage System based on Raft and RocksDB, can be use to store small files, like images.

  distributed-storagedistributed-file-systemdistributed-file-storageraftrocksdbhaystackdistributed-kv
  Language:Java 58

• cholcombe973 / rusix

  Distributed filesystem in Rust

  rustdistributed-file-system
  Language:Rust 55

• chengdedeng / yfs

  :house:超精简分布式对象存储

  jdk8raft-javaobject-storageatomixdistributed-file-systemmaven
  Language:Java 48

• dingodb / dingofs

  DingoFS is a project fork from Curve. Curve is a sandbox project hosted by the CNCF Foundation. It's cloud-native, high-performance, and easy to operate. Curve is an open-source distributed storage system for block and shared file storage.

  ai-nativecloudnativedistributed-file-systemfilesystemdistributed-cacherdma
  Language:C++ 43

• mbajer42 / ucz-dfs

  A distributed file system written in Rust.

  rustdistributed-systemshdfsfilesystemdistributed-file-system
  Language:Rust 29

• shubhamsawantsjsu / Distributed-File-Storage-System

  An open-source, scalable, decentralized, robust, heterogeneous file storage solution which is fault tolerant, replicated, distributed and lets you upload, download, and see the catalog of other cluster with low latency and LRU cache capabilities.

  distributed-file-systempython3grpc-pythonprotobuf3heartbeat-serviceraft-consensus-algorithmconsensus-algorithmlru-cacheproxy-servermultithreading
  Language:Python 27

• UnixJunkie / daft

  DAFT Allows File Transfers

  distributed-file-systemcryptographycliocaml-programuser-space
  Language:OCaml 23

• foreverbell / yfs

  Yet Another File System (yfs) based on Paxos algorithm.

  distributed-file-system
  Language:C++ 16

• ks-amit / Distributed-Database

  A travel agency app with a distributed database implemented from scratch!

  gfsdistributed-systemsdistributed-databasedistributed-storagedistributed-file-systempythonbulmabulma-cssdjango-rest-frameworkdjangopostgresqltravel-agencyheartbeat-servicefault-tolerance
  Language:Python 16

• MrCirca / zabbix-glusterfs

  Monitoring GlusterFS Discovery storage with Zabbix 3.4

  zabbix-templatesglusterfsglusterzabbix-scriptzabbix-discovery-scriptmonitoringdistributed-file-system
  Language:Python 16

• putdotio / efes

  A distributed filesystem evolved from MogileFS

  distributed-file-systemmogilefs
  Language:Go 16

• abbyssoul / libstyxe

  Implementation of 9P parser and message writer.

  9p20009pplan9styxprotocoldistributeddistributed-file-system9pfs9p2000u
  Language:C++ 15

• rajatgarg149 / BigData-Essentials-HDFS-SPARK-RDD

  big-databig-data-essentialscourseracoursera-big-datayandex-big-datasparkmapreduceapache-sparkhadoophadoop-mapreducedistributed-file-system
  Language:Jupyter Notebook 15

• PasaLab / dfs-perf

  A general performance test framework for Distributed File System

  distributed-file-systemperformance-testing
  Language:Java 14

• James-QiuHaoran / distributed-computing-platform-mapreduce

  This repository contains a simple Hadoop-like (MapReduce) distributed computing platform implemented in Java. It is extended from a course project at UIUC awarded the best Java version implementation and it's open-sourced for reference.

  mapreducehadoophadoop-mapreducedistributed-computingdistributed-file-systemmembership-managementfailure-detectiondistributed-systemscloud-computingcloud-computing-applicationshadoop-framework
  Language:Java 13

• SyncFree / antidote-fs

  A file system backed by AntidoteDB.

  filesystemdistributed-file-systemcrdt
  Language:Java 13

• ds2-lab / LambdaFS

  λFS: an elastic, high-performance, serverless-function-based metadata service for large-scale distributed file systems (ACM ASPLOS'23)

  faasfilesystemserverlessdfsdistributed-file-systemhadoop-hdfsmetadatanuclioopenwhiskserverless-computing
  Language:Java 12

• chaitanya100100 / Google-File-System

  A python implementation of Google File System

  google-file-systemgfsgrpcdistributed-file-system
  Language:Python 11

• lmco / ChaordicLedger

  The ChaordicLedger is the implementation of a design for a combination of Distributed Ledger Technology (DLT) and a Distributed File System (DFS) to create a secure, enterprise-grade platform for storing interlinked project artifacts.

  distributed-file-systemdistributed-storagedistributed-ledger-technologyblockchain-technologyipfs-blockchainobsidiansmart-contractbusiness-intelligencesoftware-supply-chaindigital-transformationdevelopment-workflow
  Language:Shell 9