DCHECK crash in `heap::base::Stack::SaveContext`
codebytere opened this issue · comments
As of https://chromium-review.googlesource.com/c/v8/v8/+/4017512, there is a worker test that crashes deterministically in Node.js: parallel/test-worker-init-failure with the following stacktrace:
Stacktrace
not ok 2980 parallel/test-worker-init-failure
---
duration_ms: 2.908
severity: fail
exitcode: 1
stack: |-
child stdout:
child stderr:
#
# Fatal error in ../../v8/src/heap/base/stack.cc, line 190
# Debug check failed: check_invariant implies IsOnCurrentStack(stack_start_).
#
#
#
#FailureMessage Object: 0x7fffe11bcff0
1: 0x55f4adeaa0b2 [/home/builduser/project/src/out/Default/electron]
2: 0x55f4aa69dff9 [/home/builduser/project/src/out/Default/electron]
3: 0x55f4aa69dc15 [/home/builduser/project/src/out/Default/electron]
4: 0x55f4a67b2aaa heap::base::Stack::SaveContext(bool) [/home/builduser/project/src/out/Default/electron]
5: 0x55f4a5c1b644 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/builduser/project/src/out/Default/electron]
6: 0x55f4a5c0fb79 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/builduser/project/src/out/Default/electron]
7: 0x55f4a5c0fc25 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/builduser/project/src/out/Default/electron]
8: 0x55f4a5bd523c v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/builduser/project/src/out/Default/electron]
9: 0x55f4a6266846 [/home/builduser/project/src/out/Default/electron]
10: 0x55f4a6266358 [/home/builduser/project/src/out/Default/electron]
11: 0x55f43feb2c78
Trace/breakpoint trap (core dumped)
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
133 !== 0
at ChildProcess.<anonymous> (/home/builduser/project/src/third_party/electron_node/test/parallel/test-worker-init-failure.js:69:12)
at ChildProcess.<anonymous> (/home/builduser/project/src/third_party/electron_node/test/common/index.js:446:15)
at ChildProcess.emit (node:events:513:28)
at ChildProcess._handle.onexit (node:internal/child_process:291:12) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 133,
expected: 0,
operator: 'strictEqual'
}
Node.js v18.12.1
I'm not quite sure why this is happening, but there is a comment in the CL:
With WASM stack switching, the main invariant is violated, as the current active stack may call JS/Blink/GC and reside somewhere else.
that indicates it may have to do with stack intricacies in a Worker context?
cc @targos
I can reproduce with a debug build of canary, but I don't know what we can do.
Here's another failure that doesn't seem related to Workers:
10:44:39 not ok 3678 sequential/test-child-process-emfile
10:44:40 ---
10:44:40 duration_ms: 0.613
10:44:40 severity: fail
10:44:40 exitcode: 1
10:44:40 stack: |-
10:44:40 node:assert:124
10:44:40 throw new AssertionError(obj);
10:44:40 ^
10:44:40
10:44:40 AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
10:44:40 + actual - expected
10:44:40
10:44:40 + '\n' +
10:44:40 + '\n' +
10:44:40 + '#\n' +
10:44:40 + '# Fatal error in ../deps/v8/src/heap/base/stack.cc, line 203\n' +
10:44:40 + '# Debug check failed: check_invariant implies IsOnCurrentStack(stack_start_).\n' +
10:44:40 + '#\n' +
10:44:40 + '#\n' +
10:44:40 + '#\n' +
10:44:40 + '#FailureMessage Object: 0xfffffa90c978\n' +
10:44:40 + ' 1: 0xaaaad0b8fcf4 node::DumpBacktrace(_IO_FILE*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 2: 0xaaaad0d53a8c [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 3: 0xaaaad0d53ab8 [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 4: 0xaaaad28ebb58 V8_Fatal(char const*, int, char const*, ...) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 5: 0xaaaad28ebb80 V8_Dcheck(char const*, int, char const*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 6: 0xaaaad1f20278 heap::base::Stack::SaveContext(bool) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 7: 0xaaaad13f84bc v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 8: 0xaaaad13c6ca0 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + ' 9: 0xaaaad13c6d48 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '10: 0xaaaad13a9bf0 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '11: 0xaaaad1357610 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawArray(int, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '12: 0xaaaad1378474 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '13: 0xaaaad13786b8 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(int, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '14: 0xaaaad0f9e450 v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, unsigned long) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '15: 0xaaaad0c83894 node::RealEnvStore::Enumerate(v8::Isolate*) const [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '16: 0xaaaad0c84f80 [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '17: 0xaaaad17b5164 v8::internal::PropertyCallbackArguments::CallPropertyEnumerator(v8::internal::Handle<v8::internal::InterceptorInfo>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '18: 0xaaaad17b73c8 v8::internal::KeyAccumulator::CollectInterceptorKeysInternal(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::InterceptorInfo>, v8::internal::KeyAccumulator::IndexedOrNamed) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '19: 0xaaaad17b8678 v8::internal::KeyAccumulator::CollectInterceptorKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::KeyAccumulator::IndexedOrNamed) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '20: 0xaaaad17c1c8c v8::internal::KeyAccumulator::CollectOwnPropertyNames(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '21: 0xaaaad17c2050 v8::internal::KeyAccumulator::CollectOwnKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '22: 0xaaaad17c2554 v8::internal::KeyAccumulator::CollectKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSReceiver>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '23: 0xaaaad17c28e4 v8::internal::FastKeyAccumulator::GetKeysSlow(v8::internal::GetKeysConversion) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '24: 0xaaaad1a18010 v8::internal::Runtime_ForInEnumerate(int, unsigned long*, v8::internal::Isolate*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + '25: 0xaaaad207018c [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 + 'Trace/breakpoint trap (core dumped)\n'
10:44:40 - ''
10:44:40 at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm-debug/test/sequential/test-child-process-emfile.js:41:10)
10:44:40 at Module._compile (node:internal/modules/cjs/loader:1218:14)
10:44:40 at Module._extensions..js (node:internal/modules/cjs/loader:1272:10)
10:44:40 at Module.load (node:internal/modules/cjs/loader:1081:32)
10:44:40 at Module._load (node:internal/modules/cjs/loader:922:12)
10:44:40 at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:82:12)
10:44:40 at node:internal/main/run_main_module:23:47 {
10:44:40 generatedMessage: true,
10:44:40 code: 'ERR_ASSERTION',
10:44:40 actual: '\n' +
10:44:40 '\n' +
10:44:40 '#\n' +
10:44:40 '# Fatal error in ../deps/v8/src/heap/base/stack.cc, line 203\n' +
10:44:40 '# Debug check failed: check_invariant implies IsOnCurrentStack(stack_start_).\n' +
10:44:40 '#\n' +
10:44:40 '#\n' +
10:44:40 '#\n' +
10:44:40 '#FailureMessage Object: 0xfffffa90c978\n' +
10:44:40 ' 1: 0xaaaad0b8fcf4 node::DumpBacktrace(_IO_FILE*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 2: 0xaaaad0d53a8c [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 3: 0xaaaad0d53ab8 [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 4: 0xaaaad28ebb58 V8_Fatal(char const*, int, char const*, ...) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 5: 0xaaaad28ebb80 V8_Dcheck(char const*, int, char const*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 6: 0xaaaad1f20278 heap::base::Stack::SaveContext(bool) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 7: 0xaaaad13f84bc v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 8: 0xaaaad13c6ca0 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 ' 9: 0xaaaad13c6d48 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '10: 0xaaaad13a9bf0 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '11: 0xaaaad1357610 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawArray(int, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '12: 0xaaaad1378474 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '13: 0xaaaad13786b8 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(int, v8::internal::AllocationType) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '14: 0xaaaad0f9e450 v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, unsigned long) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '15: 0xaaaad0c83894 node::RealEnvStore::Enumerate(v8::Isolate*) const [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '16: 0xaaaad0c84f80 [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '17: 0xaaaad17b5164 v8::internal::PropertyCallbackArguments::CallPropertyEnumerator(v8::internal::Handle<v8::internal::InterceptorInfo>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '18: 0xaaaad17b73c8 v8::internal::KeyAccumulator::CollectInterceptorKeysInternal(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::InterceptorInfo>, v8::internal::KeyAccumulator::IndexedOrNamed) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '19: 0xaaaad17b8678 v8::internal::KeyAccumulator::CollectInterceptorKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::KeyAccumulator::IndexedOrNamed) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '20: 0xaaaad17c1c8c v8::internal::KeyAccumulator::CollectOwnPropertyNames(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '21: 0xaaaad17c2050 v8::internal::KeyAccumulator::CollectOwnKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSObject>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '22: 0xaaaad17c2554 v8::internal::KeyAccumulator::CollectKeys(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::JSReceiver>) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '23: 0xaaaad17c28e4 v8::internal::FastKeyAccumulator::GetKeysSlow(v8::internal::GetKeysConversion) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '24: 0xaaaad1a18010 v8::internal::Runtime_ForInEnumerate(int, unsigned long*, v8::internal::Isolate*) [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 '25: 0xaaaad207018c [/home/iojs/build/workspace/node-test-commit-arm-debug/out/Debug/node]\n' +
10:44:40 'Trace/breakpoint trap (core dumped)\n',
10:44:40 expected: '',
10:44:40 operator: 'strictEqual'
10:44:40 }
10:44:40
10:44:40 Node.js v20.0.0-pre
10:44:40 ...
From https://ci.nodejs.org/job/node-test-commit-arm-debug/5323/nodes=ubuntu2004_debug-arm64/console
/cc @nodejs/v8
https://bugs.chromium.org/p/v8/issues/detail?id=13493 - that was fixed only yesterday.
Stack::SaveContext() is < 1 month old (edit: commit v8/v8@9554743a0ba) and interacts with experimental WASM features (--experimental_wasm_stack_switching in particular) so it's quite possible bugs lurk there.
Thanks for the heads up, Ben. I'll start a new CI job tomorrow.
The test still crashed with the same error yesterday: https://ci.nodejs.org/job/node-test-commit-arm-debug/5555/nodes=ubuntu2004_debug-arm64/testReport/junit/(root)/test/parallel_test_worker_init_failure/
There's some activity on https://bugs.chromium.org/p/v8/issues/detail?id=13493. Let's hope it will be fixed quickly.
The upstream issue is still open but my last CI run seems to say that this issue is fixed:
#244 (comment)
https://ci.nodejs.org/job/node-test-commit-arm-debug/5930/
hmm - looks like it's still happening for us on Linux: https://app.circleci.com/pipelines/github/electron/electron/62206/workflows/cbd63dd4-4578-4d5a-905d-5fc5792b6b0f/jobs/1394001
@codebytere IIUC, the job you linked uses a Chromium/V8 version from Jan 6. It's possible that the fix is more recent than that.
@codebytere Is it still an issue?
I think it's safe to say this is fixed by now? Closing, but LMK if I should reopen.