Collective pinning and composition for IPFS.
THIS SOFTWARE IS ALPHA
ipfs-cluster allows to replicate content (by pinning) in multiple IPFS nodes:
- Works on top of the IPFS daemon by running one cluster peer per IPFS node (
ipfs-cluster-service) - A
replication_factorcontrols how many times a CID is pinned in the cluster - Re-pins stuff in a different place when a peer goes down
- Provides an HTTP API and a command-line wrapper (
ipfs-cluster-ctl) - Provides an IPFS daemon API Proxy which intercepts any "pin"/"unpin" requests and does cluster pinning instead
- The IPFS Proxy allows to build cluster composition, with a cluster peer acting as an IPFS daemon for another higher-level cluster.
- Peers share the state using Raft-based consensus. Uses the LibP2P stack (
go-libp2p-raft,go-libp2p-rpc...)
This project is captained by @hsanjuan. See the captain's log for a written summary of current status and upcoming features. You can also check out the project's Roadmap for a high level overview of what's coming and the project's Waffle Board to see what issues are being worked on at the moment.
You can download pre-compiled binaries for your platform from the dist.ipfs.io website:
Note that since IPFS Cluster is evolving fast, these builds may not contain the latest features/bugfixes. Builds are updated monthly on a best-effort basis.
You can build or download an automated build of the ipfs-cluster docker container. This container runs ipfs-cluster-service and includes ipfs-cluster-ctl. To launch the latest published version on Docker run:
$ docker run ipfs/ipfs-cluster
To build the container manually you can:
$ docker build . -t ipfs-cluster
You can mount your local ipfs-cluster configuration and data folder by passing -v /data/ipfs-cluster your-local-ipfs-cluster-folder to Docker. Otherwise, a new configuration will be generated. In that case, you can point it to the right IPFS location by setting IPFS_API like --env IPFS_API="/ip4/1.2.3.4/tcp/5001".
In any of the supported Linux distros:
sudo snap install ipfs-cluster --edge(Note that this is an experimental and unstable release, at the moment)
Installing from master is the best way to have the latest features and bugfixes. In order to install the ipfs-cluster-service the ipfs-cluster-ctl tools you will need Go installed in your system and the run the following commands:
$ go get -u -d github.com/ipfs/ipfs-cluster
$ cd $GOPATH/src/github.com/ipfs/ipfs-cluster
$ make install
This will install ipfs-cluster-service and ipfs-cluster-ctl in your $GOPATH/bin folder. See the usage below.
** Remember: Start your ipfs daemon before running ipfs-cluster **
ipfs-cluster-service runs an ipfs-cluster peer:
- Initialize with
ipfs-cluster-service init- This will randomly generate a secret which should be shared among all peers.
- Run with
ipfs-cluster-service. Check--helpfor options
For more information about ipfs-cluster-service see the ipfs-cluster-service README. Also, read A guide to running IPFS Cluster for full a full overview of how cluster works.
ipfs-cluster-ctl is used to interface with the ipfs-cluster peer:
ipfs-cluster-ctl id # see peer information
ipfs-cluster-ctl pin add <cid> # Pin a CID in ipfs-cluster
ipfs-cluster-ctl pin rm <cid> # Upin a CID
ipfs-cluster-ctl ls # See current pins and allocations
ipfs-cluster-ctl status <cid> # See information from every allocation for a CID.
For information on how to manage and perform operations on an IPFS Cluster peer see the ipfs-cluster-ctl README.
IPFS Cluster nodes can be launched directly from Go. The Cluster object provides methods to interact with the cluster and perform actions.
Documentation and examples on how to use IPFS Cluster from Go can be found in godoc.org/github.com/ipfs/ipfs-cluster.
You can find more information and detailed guides:
Note: please contribute to improve and add more documentation!
TODO: Swagger
This is a quick summary of API endpoints offered by the Rest API component (these may change before 1.0):
| Method | Endpoint | Comment |
|---|---|---|
| GET | /id | Cluster peer information |
| GET | /version | Cluster version |
| GET | /peers | Cluster peers |
| POST | /peers | Add new peer |
| DELETE | /peers/{peerID} | Remove a peer |
| GET | /allocations | List of pins and their allocations (consensus-shared state) |
| GET | /allocations/{cid} | Show a single pin and its allocations (from the consensus-shared state) |
| GET | /pins | Status of all tracked CIDs |
| POST | /pins/sync | Sync all |
| GET | /pins/{cid} | Status of single CID |
| POST | /pins/{cid} | Pin CID |
| DELETE | /pins/{cid} | Unpin CID |
| POST | /pins/{cid}/sync | Sync CID |
| POST | /pins/{cid}/recover | Recover CID |
The best place to get an overview of how cluster works, what components exist etc. is the architecture.md doc.
PRs accepted.
Small note: If editing the README, please conform to the standard-readme specification.
MIT © Protocol Labs, Inc.