search-live doesn't handle s3 errors well
matthewdarwin opened this issue · comments
Similar to #182
2020-12-22T03:57:31.297Z (search-live.hub.hub.js) shutting file source, switching to live (from a file block matching) (bstream/joiningsource.go:374)
panic: unable to decode block kind EOS version 1 (5885615 payload bytes): unable to decode payload: proto: invalid field number
goroutine 618 [running]:
github.com/dfuse-io/bstream.(*Block).ToNative(0xc025246d80, 0xffffffffffffffff, 0x0)
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/bstream@v0.0.2-0.20201118191855-3c3b8b87518e/types.go:257 +0x24b
github.com/dfuse-io/dfuse-eosio/search.(*BlockMapper).Map(0xc009d65a10, 0xc025246d80, 0xc009921b30, 0x978f133, 0xc009787200, 0xc03628fad0, 0xed77364b6)
#011/tmp/build.1607621134.12959/dfuse-eosio/search/mapper.go:102 +0x40
github.com/dfuse-io/search.(*PreIndexer).Preprocess(0xc00982e5c0, 0xc025246d80, 0x0, 0x59cf54, 0x11a0000, 0xc025246d00)
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/search@v0.0.2-0.20201208141141-584c997e5937/preindexer.go:69 +0x55
github.com/dfuse-io/search/live.(*LiveBackend).SetupSubscriptionHub.func1(0xc025246d80, 0xc025246d80, 0x0, 0x0, 0x0)
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/search@v0.0.2-0.20201208141141-584c997e5937/live/setuphub.go:50 +0x41
github.com/dfuse-io/bstream.(*FileSource).streamIncomingFile(0xc009c7b700, 0xc00982eae0, 0x0, 0x0)
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/bstream@v0.0.2-0.20201118191855-3c3b8b87518e/filesource.go:228 +0x802
github.com/dfuse-io/bstream.(*FileSource).run.func1(0xc009c7b700, 0xc00923a610, 0xa, 0xc00982eae0)
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/bstream@v0.0.2-0.20201118191855-3c3b8b87518e/filesource.go:170 +0x19c
created by github.com/dfuse-io/bstream.(*FileSource).run
#011/tmp/build.1607621134.12959/go/pkg/mod/github.com/dfuse-io/bstream@v0.0.2-0.20201118191855-3c3b8b87518e/filesource.go:168 +0xae9
See also trxdb-loader
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: 2020-12-23T17:42:01.707Z (dfuse)
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: ################################################################
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: Fatal error in app trxdb-loader:
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: processing of file "0124039000" failed: block reader failed: failed reading next dbin message: unexpected EOF
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: ################################################################
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: (launcher/launcher.go:174)
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: 2020-12-23T17:42:01.707Z (dfuse) Application trxdb-loader shutdown unexpectedly, quitting (cli/start.go:161)
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: Error: unable to launch: processing of file "0124039000" failed: block reader failed: failed reading next dbin message: unexpected EOF
Dec 23 17:42:01 eos-trxdb12b dfuseeos[68826]: 2020-12-23T17:42:01.707Z (cli) dfuse (derr/cli.go:25){"error": "unable to launch: processing of file \"0124039000\" failed: block reader failed: failed reading next dbin message: unexpected
EOF"}
This seems improved recently @sduchesneau
I think this is fixed now, so closing.