Create a new signalling server for IPFS Share Files
fsdiogo opened this issue · comments
The @ipfs/gui-team has been discussing using a signalling server for the IPFS Share Files app. You can read about the why in this issue.
TL;DR: peer discovery is badly hurting the usability of the app. While this isn't fixed we thought of using a signalling server to help out.
As the app release for production is soon to be, does it makes sense to create a dedicated server for it? Or is it enough using one of the existing ones?
Hi @fsdiogo - I read through the linked issue and related issues but am not sure I fully understand the implications of running a dedicated server over using existing ones (which are those exactly?) I would be happy to go see how much the existing servers are being utilized and make a judgement as to if they can handle more traffic if that's your concern.
The existing ones that I know of are:
- https://ws-star0.ams.dwebops.pub (same as https://ws-star.discovery.libp2p.io)
- https://ws-star1.par.dwebops.pub
- https://ws-star2.sjc.dwebops.pub
see how much the existing servers are being utilized and make a judgement as to if they can handle more traffic
My concern is exactly that. We're hoping that this app generates a lot of traffic in the future, so we want to make sure that the servers handle that, and we have the problem that when a server is down, it just plain crashes js-ipfs, so having the server up is key or else the app won't work at all.
We talked this through on our weekly catch up and @eefahy and I had a good conversation about it.
The short version is we need create documentation and a story around share that clearly explains the goals / perf expectations that users should have, and the ipfs config and any points of centralisations or infrastructure required to meet those expectations today.
With that we can drill in to what our expectations are of the preload nodes. @eefahy made the good point that if the preload nodes are working as intended, we may not need to add a ws-star discovery and transfer node as well. We can use this project as an opportunity to work with infra to help define things like "what is the average and minimum time a block will be available from a preload node" which can be used to define the hosting strategy for the preload nodes, and how do we signal to app developers when the preload nodes are experinencing a spike in trafffic that might push that time down (preload will pin blocks until they exceed their configured storageMax which will trigger GC, which will clear out older blocks, so at busy times the min storage time could drop below the advertised minimum) or what should happen if those nodes are offline.
It works both ways too; infra can call out issues with the current IPFS architecture that make it harder to set up good Site Reliabilty practices (clustering, load balance, decoupling of resources from specific instances looking at you peerId in the preload node definitions)
The goal is not to release share.ipfs.io as soon as possible, but to make sure that when we do it is an exemplary IPFS based service which can be used as an educational resource, and a set of recommendations to app builders and the community. It's worth doing this well, even it means delaying our release for it.
@fsdiogo I'll work with you on an outline for this work, and we'll ping @eefahy and @daviddias for feedback on it as we go.