distributed-storage

There are 67 repositories under distributed-storage topic.

• seaweedfs

  seaweedfs / seaweedfs

  SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  blob-storagecloud-drivedistributed-file-systemdistributed-storagedistributed-systemserasure-codingfusehadoop-hdfshdfskubernetesobject-storageposixreplications3s3-storageseaweedfstiered-file-system
  Language:Go 20995
• ceph

  ceph / ceph

  Ceph is a distributed object, block, and file storage platform

  storagesoftware-defined-storagedistributed-storages3block-storagedistributed-file-systemobject-storenfshighly-availableiscsicloud-storagekuberneteshdfssmbhigh-performancefuseposixerasure-codingreplicationnvme-over-fabrics
  Language:C++ 13195

• lni / dragonboat

  A feature complete and high performance multi-group Raft library in Go.

  raftpaxosgolanggodistributed-systemsraft-protocolraft-algorithmconsensusreplicated-state-machinesdistributed-storagedistributed-consensus
  Language:Go 4933

• cubefs / cubefs

  cloud-native file store

  distributed-storagecncfkubernetescloud-storageerasure-codinghybrid-cloudai-native-storageobject-storagecloud-native-storagedistributed-file-systemdata-orchestrationfuse
  Language:Go 4310

• baidu / braft

  An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

  raft-consensus-algorithmraftraft-protocolraft-cppraft-cxxdistributed-storagedistributed-consensusstate-machine-replication
  Language:C++ 3781

• storj / storj

  Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

  distributeddistributed-storagedistributed-systemsgolangobject-storageopen-sourcestorage
  Language:Go 2864

• pravega / pravega

  Pravega - Streaming as a new software defined storage primitive

  streamingstreaming-datadistributed-storagereal-time-datadata-ingestion
  Language:Java 1965

• moosefs / moosefs

  MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

  software-defined-storageposixfilesystemfile-systemdistributed-file-systemclusteringdistributed-storagedistributed-computingfusebig-datasnapshotstorage-tieringhigh-availabilityscalabilitystoragemoosefshadoopposix-compliantpetabyteditributed-systems
  Language:C 1583

• leo-project / leofs

  The LeoFS Storage System

  datalakedistributed-file-systemdistributed-storageerlangleofsnfsnfs-servers3s3-storage
  Language:Erlang 1538

• eclipse-zenoh / zenoh

  zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

  networkingprotocolmessagingstorageedge-computingiotroboticsembeddedros2zenohrustdistributed-computingdistributed-storagedistributed-systemsgeo-distributed-storagesnetwork-programmingrust-lang
  Language:Rust 1237

• eventql / eventql

  Distributed "massively parallel" SQL query engine

  databasesqltimeseriesmppcppcpp11analyticscolumnar-storagestreamingeventqldistributeddistributed-databasedistributed-storagedistributed-sql
  Language:C++ 1177

• wenweihu86 / raft-java

  Raft Java implementation which is simple and easy to understand.

  distributed-file-systemdistributed-storageraftraft-algorithmraft-consensusraft-consensus-algorithmraft-javaraft-protocol
  Language:Java 1148

• sheepdog / sheepdog

  Distributed Storage System for QEMU

  qemuqemu-kvmiscsiiscsi-targetblockdevicestoragedistributed-storagecvirtualizationvmmsoftware-defined-storage
  Language:C 977

• Qihoo360 / huststore

  High-performance Distributed Storage

  architecturebenchmarkredisdatabase-enginenginxmessage-queuedistributed-storage
  Language:C 825

• zuoyebang / bitalostored

  Bitalostored is a high-performance distributed storage system, core engine based on bitalosdb(self-developed), compatible with Redis protocol.

  bitalosdbdatabasedistributed-storagehigh-performancekvstorenosqlredisstorage-engine
  Language:Go 812

• gojek / awesome-distributed-systems

  Awesome list of distributed systems resources

  awesomeawesome-listcomputer-engineeringcomputer-sciencedistributeddistributed-computingdistributed-databasedistributed-storagedistributed-systems
  798

• daos-stack / daos

  DAOS Storage Stack (client libraries, storage engine, control plane)

  distributed-storageobject-storekvstorespdknvme-storagesoftware-defined-storagehyperconvergedpmdkhyperscaledaosstorage-as-a-service
  Language:C 695

• Seagate / cortx

  CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

  s3-storagesoftware-defined-storagestoragestorage-apiobject-storageobject-storage-serviceobjectstorageobjectstoreopensourceopen-sourcehacktoberfesthacktoberfest2020cortx-communityinclusivityhackathonss3distributed-systemsdistributed-storagebigdatabig-data
  Language:Jupyter Notebook 631

• douban / gobeansdb

  Distributed object storage server from Douban Inc.

  memcachedkvstoreobject-storagedistributed-storage
  Language:Go 449
• good-karma-kit

  ArchiveBox / good-karma-kit

  😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

  dockerdocker-composedistributed-computinggood-karmatori2pipfsstorjsiaboincfoldingathomearchivewarriorzimfarmkiwixarchiveboxpywbinternet-archivingdistributed-storage
  293

• diguage / mysql-notes

  MySQL 学习笔记

  distributed-databasedistributed-storagedistributed-systemsdistributed-transactionsmysqlmysql-databasemysql-servertransaction-management
  Language:Shell 266

• chrislusf / vasto

  A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

  distributed-storagekey-valuerocksdbreplication
  Language:Go 257

• mentalfaculty / LLVS

  Low-Level Versioned Store

  swiftswiftuisyncsynchronizationdistributed-storage
  Language:Swift 208

• matrixorigin / matrixcube

  Fundamental Building Block for Elastic Storage With Strong Consistency and Reliability

  distributed-systemscloud-nativeconsensusraftpaxosdistributed-storagego
  Language:Go 201

• SUSE / DeepSea

  A collection of Salt files for deploying, managing and automating Ceph.

  cephsaltpythonsoftware-defined-storagedistributed-storagedeployment-system
  Language:Python 161

• zuoyebang / bitalosdb

  Bitalosdb is a high-performance KV storage engine.

  databasedistributed-storagehigh-performancekvstorenosqlstorage-engineio-architecture
  Language:Go 153

• freakmaxi / kertish-dfs

  Kertish-dfs is a simple distributed storage platform, implements file storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dfs aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

  data-farmdfsdistributed-file-storagedistributed-file-systemdistributed-storagescalable
  Language:Go 106

• layr-team / Layr

  A decentralized (p2p) file storage system built atop Kademlia DHT that enforces data integrity, privacy, and availability through sharding, proofs of retrievability, redundancy, and encryption, with smart-contract powered incentive scheme

  tcpfile-storagep2pnodejsstreamskademliadhtdecentralized-storagedecentralizeddistributed-systemsdistributed-storageproof-of-retreivalblockchainshardingstellar-networkstellar-lumensblockchain-networkpeer-to-peerp2p-nodesmart-contracts
  Language:JavaScript 91

• lustre / lustre-release

  Mirror of official Lustre development repository http://git.whamcloud.com/

  filesystemfile-systemhpc-clusterlinux-kernel-moduleposixdistributed-storage
  Language:C 90

• pitchpoint-solutions / sfs

  The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

  distributed-storagedistributed-systemsvertxrxjavaelasticsearchopenstack-swiftdistributed-file-systemobject-storagejava
  Language:Java 83

• CDDSCLab / DMCL-2018

  电子科技大学分布式存储与计算实验室2018新生学习资料整理

  distributed-systemsdistributed-storageuestc
  Language:JavaScript 78

• zettastor / dbs

  ZettaStor DBS provides enterprise-level business storage solutions with high availability, high performance, easy expansion and easy maintenance for large-scale virtualization, private cloud and container environments.

  clusterdistributed-storagescalabilitystoragepengyun
  72

• aganesan4 / CORDS

  File-system fault injection framework for distributed storage systems

  fault-injectionfilesystemdistributed-storage
  Language:Python 60

• vesoft-inc / nebula-storage

  A distributed consistent graph storage. This is an archived repo for v2.5 only, from 2.6.0 +, NebulaGraph switched back to https://github.com/vesoft-inc/nebula

  graph-databasegraph-storagedistributed-storagenebula-graph
  Language:C++ 59

• wenweihu86 / distkv

  Distributed KV Storage System based on Raft and RocksDB, can be use to store small files, like images.

  distributed-storagedistributed-file-systemdistributed-file-storageraftrocksdbhaystackdistributed-kv
  Language:Java 56

• Oaklight / Vesper-raft

  A fault-tolerant distributed key-value store using the Raft consensus algorithm.

  raftdistributed-storagekey-valuepython3consensus-algorithm
  Language:Python 48