prb faild to sync beyond block 411243 (StateRootMismatch)

[Tianji Wu]

When syncing with lifecycle, got the following error with all workers.

{"message":"StateRootMismatch block=411243 expected=0x3e37ab9adfb2e3a5a91c8f08ec4ec65b69e73d6ace0f8fbf10730f88fa1d67c2 actual=0xfd8d9d477dd9920184c75fa9d733be5237c8059017fe20d99a1703ef241556a1"}

The root cause is with polkadot.js: polkadot-js/api#4208
This issue results in an incorrect db cache with prb fetch.

In the db cache, we use the following type definitions:

...
  BlockHeaderWithChanges: {
    blockHeader: 'Header',
    storageChanges: 'StorageChanges',
  },
  StorageCollection: 'Vec<(Vec<u8>, Option<Vec<u8>>)>',
  ChildStorageCollection: 'Vec<(Vec<u8>, StorageCollection)>',
  StorageChanges: {
    mainStorageChanges: 'StorageCollection',
    childStorageChanges: 'ChildStorageCollection',
  },
...

Here StorageCollection uses the structure of Option<Vec<u8>>, which has the issue.

In block #411243, the last two entries of mainStorageChanges are (as returned by node with RPC):

[
    "0xfcf02af62786f6514fecf014aae642792bb3e04aed192a3286411d98f68af3acbb2e84115db0cfa60e2933668f547fdc9e31f124fb1d2fea2be0fbd6182aab2321ebec4359ef476a",
    "0x"
],
[
    "0xfcf02af62786f6514fecf014aae642792bb3e04aed192a3286411d98f68af3acd39a2d2fb7c81fd224af71d17ed2ac5ae46faf8402927afadb02a34c42607507c6d3536a7289020b",
    "0x"
]

However, polkadot.js will decode these two entries into the following, which is wrong.

[
    "0xfcf02af62786f6514fecf014aae642792bb3e04aed192a3286411d98f68af3acbb2e84115db0cfa60e2933668f547fdc9e31f124fb1d2fea2be0fbd6182aab2321ebec4359ef476a",
    null
],
[
    "0xfcf02af62786f6514fecf014aae642792bb3e04aed192a3286411d98f68af3acd39a2d2fb7c81fd224af71d17ed2ac5ae46faf8402927afadb02a34c42607507c6d3536a7289020b",
    null
]

I tried manually fixing up the binary (SCALE) representation of the db cache, and successfully synced this block to pRuntime.

[Tianji Wu]

This issue is with polkadot.js v6.4.2 - v6.8.2-2 (latest version)

[h4x3rotab]

Fabulous! It definitely deserve a tip! @shelvenzhou

[Tianji Wu]

I proposed a tip for myself 😊

[Krhougs]

Since the Option<Vec<T>> encoding behavior is not consistent with Rust side, we can't promise that any trick on JS side be stable and reliable.

A fix will be done in the on-chain RPC interface along with the Substrate 0.9.12 upgrade to Khala's codebase.

[ilgio]

I have same problem

[Krhougs]

Fixed in 9f63d93 with latest khala-node

Will release after testing.

[ilgio]

closed but not solved?