Giters

apple / foundationdb

FoundationDB - the open source, distributed, transactional key-value store

Home Page:https://apple.github.io/foundationdb/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

rare/BlobGranuleRanges.toml hits assertion failure in Ratekeeper code

sfc-gh-jshim opened this issue · comments

commented

Code location and commit hash:

foundationdb/fdbserver/Ratekeeper.actor.cpp

Line 1025 in 36e8e5a

ASSERT(!g_network->isSimulated() || limits->bwLagTarget != SERVER_KNOBS->TARGET_BW_LAG ||

Ensemble ID: 20230120-033919-nightly_correctness_main_x86_64-56a90c84c13adea5

Trace snippet:

  <InternalError Severity="40" ErrorKind="BugDetected" Time="884.885138" DateTime="2023-01-20T05:16:23Z" Type="InternalError" Machine="2.1.1.3:1" ID="0000000000000000" Error="internal_error" ErrorDescription="An internal error occurred" ErrorCode="4100" FailedAssertion="!g_network-&gt;isSimulated() || limits-&gt;bwLagTarget != SERVER_KNOBS-&gt;TARGET_BW_LAG || now() &lt; FLOW_KNOBS-&gt;SIM_SPEEDUP_AFTER_SECONDS + SERVER_KNOBS-&gt;BW_RK_SIM_QUIESCE_DELAY" File="/home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbserver/Ratekeeper.actor.cpp" Line="1026" ThreadID="1304401877546049582" Backtrace="addr2line -e fdbserver.debug -p -C -f -i 0x4328ea1 0x42700ba 0x280e1f6 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555" LogGroup="default" Roles="CP,CS,DD,MS,RK,RV,SS"/>
  <SystemError Severity="40" ErrorKind="Unset" Time="884.885138" DateTime="2023-01-20T05:16:23Z" Type="SystemError" Machine="2.1.1.3:1" ID="0000000000000000" Error="internal_error" ErrorDescription="An internal error occurred" ErrorCode="4100" ThreadID="1304401877546049582" Backtrace="addr2line -e fdbserver.debug -p -C -f -i 0x4328ea1 0x42706ef 0x42700cd 0x280e1f6 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555" LogGroup="default" Roles="CP,CS,DD,MS,RK,RV,SS"/>
  <Crash Severity="40" ErrorKind="BugDetected" Time="884.885138" DateTime="2023-01-20T05:16:23Z" Type="Crash" Machine="2.1.1.3:1" ID="0000000000000000" Signal="6" Name="Aborted" Trace="addr2line -e fdbserver.debug -p -C -f -i 0x42700cd 0x280e1f6 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555" ThreadID="1304401877546049582" Backtrace="addr2line -e fdbserver.debug -p -C -f -i 0x4328ea1 0x42fea82 0x7fbd3a4c9630 0x42700cd 0x280e1f6 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555" LogGroup="default" Roles="CP,CS,DD,MS,RK,RV,SS"/>
  <WarningLimitExceeded Severity="30" WarningCount="334"/>
  <TestUnexpectedlyNotFinished Severity="40"/>
  <StdErrOutput Severity="40" Output="Assertion !g_network-&gt;isSimulated() || limits-&gt;bwLagTarget != SERVER_KNOBS-&gt;TARGET_BW_LAG || now() &lt; FLOW_KNOBS-&gt;SIM_SPEEDUP_AFTER_SECONDS + SERVER_KNOBS-&gt;BW_RK_SIM_QUIESCE_DELAY failed @ /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbserver/Ratekeeper.actor.cpp 1026:"/>
  <StdErrOutput Severity="40" Output=" addr2line -e fdbserver.debug -p -C -f -i 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555"/>
  <StdErrOutput Severity="40" Output="SIGNAL: Aborted (6)"/>
  <StdErrOutput Severity="40" Output="Trace: addr2line -e fdbserver.debug -p -C -f -i 0x42700cd 0x280e1f6 0x28340cd 0x2827507 0x1a51858 0x420bfe5 0x2d561c7 0x7fbd3a10e555"/>
</Test>

Assigning to @sfc-gh-jslocum for being a test owner and having modified the assertion lines with commit 9721de7

commented

This test passes when increasing BW_RK_SIM_QUIESCE_DELAY
Given that PR #9052 increases the timeout and makes fixes around blob worker throttling, I'm inclined to just close this, and if a similar error appears after merging, to revisit then.