server only serves chunks in forward direction

Question

server only serves chunks in forward direction

ecdsa opened this issue 7 years ago · comments

This has been introduced in 04e7dbe, as a fix for peer discovery, when misconfigured servers returned a mixture of testnet and mainnet servers to the client.

The client now requests a checkpoint (default checkpoint is the genesis block) when it initiates a connection, and it closes the connection if the returned header fails to pass the checkpoint. Therefore this fix is not needed anymore.

Clients may need to request chunks in reverse order if there is a hard fork and they want to switch to the other branch. Commit 04e7dbe prevents them from fetching the other branch.

ThomasV · Answer 1 · Thu Mar 30 2017 00:03:31 GMT+0800 (China Standard Time)

to reproduce the issue:

connect to testnet.hsmith.com (hardforking testnet server)
configure a checkpoint, in order to follow the forking chain. (e.g. at 1100700)
remove the checkpoint (set it to zero)

After the checkpoint is removed, the client tries to download the other (longer) chain, but servers refuse to serve it.

Neil · Answer 2 · Thu Mar 30 2017 05:17:49 GMT+0800 (China Standard Time)

This was a huge bandwidth exhaustion issue because of prior client behaviour,and the reason JWU42 still doesn't run a public testnet server. I'm not willing to remove it until the number of older clients shrinks considerably (which is the first good version?)
Is it possible to request in the forward direction from the client by truncating the bad header set?

Johann Bauer · Answer 3 · Thu Mar 30 2017 05:24:10 GMT+0800 (China Standard Time)

Maybe you could enable that for newer clients only?

ThomasV · Answer 4 · Thu Mar 30 2017 11:17:37 GMT+0800 (China Standard Time)

@kyuupichan In that case please allow a longer interval than 2 chunks. (20 should be plenty)
It is not possible to request in the forward direction because the client does not know where to start requesting from.

ThomasV · Answer 5 · Thu Mar 30 2017 11:22:23 GMT+0800 (China Standard Time)

the first "good" version is 2.8.3

Neil · Answer 6 · Fri Mar 31 2017 07:20:51 GMT+0800 (China Standard Time)

I'll have a look at enabling this, perhaps with a larger limit, for >= 2.8.3 clients this w/e. I think the logic on the client side can be improved too. This checkpoints - is there an associated height, or does the client just have a hash?

Assuming stepping backwards is permitted, any reason not to binary search for the chunk which first differs?

ThomasV · Answer 7 · Fri Mar 31 2017 13:33:16 GMT+0800 (China Standard Time)

Checkpoints do have an associated height, as you might have seen in the screenshot I posted on twitter.
I am working on a binary search, but I do not think it will be ready for the next release.

Neil · Answer 8 · Sat Apr 01 2017 10:10:37 GMT+0800 (China Standard Time)

@ecdsa perhaps we should consider an improved protocol where the client passes a block locator like in bitcoind, starting at current height, but we could require it to go back to genesis. I'm not sure if either client or server would benefit from it being (hash, height) pairs or simply hashes as in bitcoind.
The server could return the number of entries that it agrees with (obviously, counting from the back). 0 would mean wrong genesis. Any other reply would allow the client to know reasonably well where to start requesting chunks (or it could refine its request if appropriate)