node websocket subscription has 15s delay and missing data

[ella]

Problem

Using websocket connections we observed 2 behaviours and these 2 issues.

Issue 1

Websocket subscription has an average of 15s plus delay for finalised commitment.
You can use this gist to replicate.

Issue 2

Websocket subscription has missing data.
You can use this gist to replicate.
In this gist, you can add multiple nodes in the array. Script will run for 1 min and output a table, it filter through unique txn hashes against each node and output the missing logs for that node with a timestamp.

I noted the behaviour was caused when node was lagging slightly then sync 2-3 slots very quickly, and for data stream through websocket doesn't seem to be able to handle it gracefully, hence the subscription will just miss some data.

[github-actions]

This repository is no longer in use. Please re-open this issue in the agave repo: https://github.com/anza-xyz/agave