object-storage

There are 53 repositories under object-storage topic.

• seaweedfs

  seaweedfs / seaweedfs

  SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

  blob-storagecloud-drivedistributed-file-systemdistributed-storagedistributed-systemserasure-codingfusehadoop-hdfshdfskubernetesobject-storageposixreplications3s3-storageseaweedfstiered-file-system
  Language:Go 27353

• parse-community / parse-server

  Parse Server for Node.js / Express

  baasbackendfile-storagegraphqlgraphql-apigraphql-relaygraphql-serverhacktoberfestmbaasmongodbnodejsnotificationsobject-storageparse-platformparse-serverpostgresrelayrest-api
  Language:JavaScript 21313
• juicefs

  juicedata / juicefs

  JuiceFS is a distributed POSIX file system built on top of Redis and S3.

  bigdatacloud-nativedistributed-systemsfilesystemgogolanghdfsobject-storageposixrediss3storage
  Language:Go 12384

• rustfs / rustfs

  🚀 RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.

  amazon-s3bigdatacloud-nativefilesystemminioobject-storageobjectstoragerusts3
  Language:Rust 11168
• automq

  AutoMQ / automq

  AutoMQ is a diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.

  apache-kafkacloudcloud-firstcloud-nativedisklesskafkamessagingminiominio-storageobject-storages3serverlessstreaming
  Language:Java 7642

• awslabs / mountpoint-s3

  A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.

  awsfilesystemfuseobject-storages3
  Language:Rust 5479

• cubefs / cubefs

  cloud-native distributed storage

  ai-native-storagecloud-native-storagecloud-storagecncfdata-orchestrationdistributed-file-systemdistributed-storageerasure-codingfusehybrid-cloudkubernetesobject-storage
  Language:Go 5304
• lakeFS

  treeverse / lakeFS

  lakeFS - Data version control for your data lake | Git for data

  data-engineeringdata-versioninggoobject-storagedata-lakeaws-s3data-qualityazure-blob-storagegoogle-cloud-storagegolanggit-for-datalakefsapache-sparkapache-sparksqlhadoop-filesystemdatalakedatalakesdata-version-controlazure-storage
  Language:Go 4966

• storj / storj

  Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

  distributeddistributed-storagedistributed-systemsgolangobject-storageopen-sourcestorage
  Language:Go 3195

• slatedb / slatedb

  A cloud native embedded storage engine built on object storage.

  databaseembedded-databaselsm-treeobject-storagerocksdbruststorage-engine
  Language:Rust 2440
• cloudserver

  scality / cloudserver

  Zenko CloudServer, an open-source Node.js implementation of the Amazon S3 protocol on the front-end and backend storage capabilities to multiple clouds, including Azure and Google.

  aws-s3dockermultiple-backendsstorageclouds3-storagecloud-storagecloud-nativeobject-storagenodejsjavascriptzenkoartesca
  Language:JavaScript 1849

• deuxfleurs-org / garage

  (Mirror) S3-compatible object store for small self-hosted geo-distributed deployments. Main repo: https://git.deuxfleurs.fr/Deuxfleurs/garage

  object-storagerusts3
  Language:Rust 1697

• NVIDIA / aistore

  AIStore: scalable storage for AI applications

  object-storageetl-offloadlinear-scalabilitybatch-jobskubernetesdistributed-storagehigh-availabilityhigh-performanceml-trainingmulti-cloudmultipart-uploads3-compatible
  Language:Go 1625

• supabase / storage

  S3 compatible object storage service that stores metadata in Postgres

  gotrueobject-storagepostgress3
  Language:TypeScript 1137
• nitrite-java

  nitrite / nitrite-java

  NoSQL embedded document store for Java

  nosqlembedded-databasedocumentdbandroidmobile-databasedatabaseobject-storageobject-database
  Language:Java 878

• open-io / oio-sds

  High Performance Software-Defined Object Storage for Big Data and AI, that supports Amazon S3 and Openstack Swift

  storageobject-storagecpythonsoftware-defined-storagelightweightgolangopen-sourceflexibleobjectstorages3-storageopenstack-swift
  Language:Python 727
• Zenko

  scality / Zenko

  Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.

  multi-cloudaws-s3azure-storagegoogle-cloudscalityobject-storagedocker-swarmzenkohybrid-cloudkuberneteshelmcloud-nativeprometheusgrafanakafkazookeepermongodbredisartesca
  Language:Gherkin 631

• Seagate / cortx

  CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

  s3-storagesoftware-defined-storagestoragestorage-apiobject-storageobject-storage-serviceobjectstorageobjectstoreopensourceopen-sourcehacktoberfesthacktoberfest2020cortx-communityinclusivityhackathonss3distributed-systemsdistributed-storagebigdatabig-data
  Language:Jupyter Notebook 620

• paradedb / pg_analytics

  DuckDB-powered data lake analytics from Postgres

  analyticsarrowbig-datacolumnardatabasedatafusiondatalakedeltalakeduckdbiceberglakehouselakehouse-platformobject-storageolapparadedbparquetpostgrespostgresqlrealtime-analyticssql
  Language:Rust 530

• Just-Moh-it / Mockoops

  Create 💎 beautiful animations from boring screen recordings within ⏳ seconds

  lambdamockupmockup-generatorobject-storageremotionrenderanimationbrowsercanvaspwareactreactjsscreen-recording
  Language:JavaScript 467

• douban / gobeansdb

  Distributed object storage server from Douban Inc.

  memcachedkvstoreobject-storagedistributed-storage
  Language:Go 450

• frugalos / frugalos

  Frugal Object Storage

  rustdistributed-object-storeobject-storage
  Language:Rust 406

• minio / dsync

  A distributed sync package.

  distributed-locksdistributed-systemsquorumlockingcloud-storageobject-storage
  Language:Go 399

• ebogdum / callfs

  CallFS is an ultra-lightweight, high-performance REST API filesystem that provides precise Linux filesystem semantics over various backends including local filesystem, Amazon S3, and distributed peer networks

  filesystemgogolangobject-storagerest-apirestapirestfulrestful-apis3storagestorage-api
  Language:Go 385

• mosuka / phalanx

  Phalanx is a cloud-native distributed search engine that provides endpoints through gRPC and traditional RESTful API.

  cloud-nativedistributedenginegogolanggossip-protocolgrpcobject-storagerestful-apisearch
  Language:Go 373

• infinit / infinit

  The Infinit policy-based software-defined storage platform.

  infinitstorageplatformfault-toleranceresiliencedecentralizeddistributedpeer-to-peerkey-valuefilesystemobject-storageblock-devicepoliciesscalabilityfusenfsiscsinbdldaps3
  367

• lithops-cloud / lithops

  A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀

  big-databig-data-analyticscloud-computingdata-processingdistributedkubernetesmulticloudmultiprocessingobject-storageparallelpythonserverlessserverless-computingserverless-functions
  Language:Python 352

• qinguoyi / osproxy

  对象存储分布式代理(object storage distrbuted proxy)，支持Docker一键部署

  s3object-storageginproxy
  Language:Go 296

• helyim / helyim

  seaweedfs implemented in pure Rust

  object-storages3iouringdpdkrdmaspdkkernel-bypasshdfswebdaverasure-codingcuda
  Language:Rust 204

• uber-go / dosa

  DOSA is a data object abstraction layer

  abstractioncassandradosaobject-storagestorage
  Language:Go 203

• thanos-io / objstore

  Go module providing unified interface and efficient clients to work with various object storage providers until like GCS, S3, Azure, SWIFT, COS and more.

  azure-storagegcsminioobject-storages3swift
  Language:Go 168

• durner / AnyBlob

  AnyBlob - A Universal Cloud Object Storage Download Manager Built For Cost-Throughput Optimal Analytics!

  blob-storagecloudcloud-storagedownloaderobjectobject-storagestorage
  Language:C++ 139

• EMCECS / ECS-CommunityEdition

  ECS Community Edition "Free & Frictionless"

  ansibleclouddellemcecsemcobjectobject-storagestorage
  Language:JavaScript 136

• IBM / jupyterlab-s3-browser

  A JupyterLab extension for browsing S3-compatible object storage

  jupyterlab-extensionjupyterlabs3object-storage
  Language:TypeScript 131

• datopian / r2-bucket-uploader

  Cloudflare R2 bucket File Uploader with multipart upload enabled. Tested with files up to 10 GB size.

  blobblob-storagebucketcloudflaredata-lakeobject-storager2s3
  Language:TypeScript 128

• noobaa / noobaa-operator

  Operator for NooBaa - object data service for hybrid and multi cloud environments :cloud: :wrench:

  crdkubernetesnoobaamulti-cloud-environmentsobject-storageocsopenshiftk8soperatorcli
  Language:Go 118