feat: improve offline recoverability of Filter

[Sasha]

This is a feature request

Problem

Currently once js-waku looses and then recovers connection (either Internet or to all peers) - existing Filter subscriptions misbehave.

Proposed Solutions

If internet connection was lost:

1. redial peers to which subscription was created;
2. ping and if fails - try to create a new one;

If peers were disconnected:

1. simply trying to redial should help;

These are things that helped me to recover subscriptions during my local experimentation.

Notes

Part of waku-org/pm#143

[Sasha]

@danisharora099 would be happy to hear your opinion on the approach

[Danish Arora]

@danisharora099 would be happy to hear your opinion on the approach

Thanks for documenting @weboko.

I'd see these two cases under the same umbrella, as application is not aware if it's a peer disconnection or an internet connection loss -- both of them need to be inferred by recurring Filter pings failing:

• upon failure, check if we still have a connection open with the peer
• if not, create new filter subscriptions
  • also attempt to dial the lost connection with the same peer
• if no peers are available for a filter subscription, keep retrying while toggling the node status to offline (ConnectionManager) as it means that there are no available Filter connect

--
PS: perhaps the online/offline state in ConnectionManager can be extended to be per protocol instead.

[Danish Arora]

@weboko I see this is in progress. I'm curious of how this will be actioned. Can you help share your POV to this?
To me it looks like a followup after #2002 is implemented, where we have access to the state of peers used by each protocol, and in this case Filter.

redial peers to which subscription was created;
ping and if fails - try to create a new one;

[Danish Arora]

Note: the Filter pings and renewals is a part of #2002. The scope of this issue is solving for js-waku's disconnection