Allow multiple IPFS nodes to advertise their CIDs into IPNI through the same instance of index-provider
ischasny opened this issue · comments
index-provider
allows IPFS nodes to advertise their CIDs to IPNI via Delegated Routing protocol. index-provider
is supposed to be run as a sidecar to the IPFS node that it is publishing for. Such architecture is inefficient for IPFS clusters with more than one node as each node has to have its own instance of index-provider
running. There are no limitations in the protocol itself that would prevent multiple IPFS nodes from sharing the same instance of index-provider
.
As a part of this work:
- enable
index-provider
to support multiple nodes sharing a single instance via HTTP Delegated Routing; - optimise
index-provider
in-memory data structure that is used to facilitate CID expiration. Right now the amount of RAM required is linearly correlated to the number of active CIDs which will be a bottleneck for larger deployments.
An alternative to that approach could be using Elastic IPFS as it supports direct publishing to IPNI. The downside however is that it has a complex deployment architecture and is tight to AWS only.
Related thread.