github.com/cockroachdb/pebble/internal/metamorphic: TestMeta failed
cockroach-teamcity opened this issue · comments
github.com/cockroachdb/pebble/internal/metamorphic.TestMeta failed with artifacts on refs/heads/master @ b6e563f67d90:
// INFO: [JOB 1311] sstable deleted 001133
// INFO: [JOB 1316] ingesting: sstable created 001141
// INFO: [JOB 1311] sstable deleted 001134
// INFO: [JOB 1319] compacting(move) multilevelL1 [001140] (707B) Score=21350.00 + L2 [] (0B) Score=0.00 + L3 [] (0B) Score=7001.12; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1316] ingested L0:001141 (876B), L2:001142 (782B)
db1.IngestExternalFiles(external1, "iiinlcyczdt" /* start */, "iiiqvpbm" /* end */, "@81" /* syntheticSuffix */, "iii" /* syntheticPrefix */, external0, "pwdtmyznv" /* start */, "vikxly" /* end */, "" /* syntheticSuffix */, "" /* syntheticPrefix */, external1, "pqyh" /* start */, "pwdtmyznv" /* end */, "@82" /* syntheticSuffix */, "" /* syntheticPrefix */) // <nil> #6604
snap43.Get("xekbcbic@8") // [""] pebble: not found #6607
db1.Set("pwdtmyznv@79", "tpirumy") // <nil> #6609
// INFO: [JOB 1320] ingested L0:001143 (782B)
db1.IngestExternalFiles(external1, "ypwdtmyznv" /* start */, "yqvpbm" /* end */, "" /* syntheticSuffix */, "y" /* syntheticPrefix */, external0, "ivzc" /* start */, "pwdtmyznv" /* end */, "" /* syntheticSuffix */, "" /* syntheticPrefix */) // <nil> #6610
db1.Delete("fetbq@8") // <nil> #6612
db1.Merge("etjrovdge@9", "h") // <nil> #6614
db1.Set("ftlrhnihbwj@24", "lyjeshepdmoitbemdtdo") // <nil> #6620
// INFO: [JOB 1322] compacting(rewrite) L0 [001141] (876B) Score=0.00 + L0 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1323] compacting(rewrite) L0 [001128] (782B) Score=0.00 + L0 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1324] compacting(rewrite) L0 [001143] (782B) Score=0.00 + L0 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1324] compacting: sstable created 001144
// INFO: [JOB 1319] compacted(move) multilevelL1 [001140] (707B) Score=21350.00 + L2 [] (0B) Score=0.00 + L3 [] (0B) Score=7001.12 -> L3 [001140] (707B), in 0.0s (0.0s total), output rate 143MB/s
// INFO: [JOB 1324] compacted(rewrite) L0 [001143] (782B) Score=0.00 + L0 [] (0B) Score=0.00 -> L0 [001144] (737B), in 0.0s (0.0s total), output rate 1.7MB/s
// INFO: [JOB 1323] compacting: sstable created 001145
// INFO: [JOB 1323] compacted(rewrite) L0 [001128] (782B) Score=0.00 + L0 [] (0B) Score=0.00 -> L0 [001145] (746B), in 0.0s (0.0s total), output rate 222KB/s
// INFO: [JOB 1326] compacting(move) multilevelL3 [001136] (1.1KB) Score=7380.21 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1326] compacted(move) multilevelL3 [001136] (1.1KB) Score=7380.21 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=0.00 -> L5 [001136] (1.1KB), in 0.0s (0.0s total), output rate 175MB/s
// INFO: [JOB 1327] compacting(move) multilevelL3 [001137] (1.2KB) Score=5998.96 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=0.58; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1327] compacted(move) multilevelL3 [001137] (1.2KB) Score=5998.96 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=0.58 -> L5 [001137] (1.2KB), in 0.0s (0.0s total), output rate 364MB/s
// INFO: [JOB 1328] compacting(move) multilevelL3 [001138] (1.2KB) Score=4276.04 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=4.57; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1328] compacted(move) multilevelL3 [001138] (1.2KB) Score=4276.04 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=4.57 -> L5 [001138] (1.2KB), in 0.0s (0.0s total), output rate 434MB/s
// INFO: [JOB 1329] compacting(move) multilevelL3 [001139] (1.1KB) Score=2636.46 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=6.98; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1329] compacted(move) multilevelL3 [001139] (1.1KB) Score=2636.46 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=6.98 -> L5 [001139] (1.1KB), in 0.0s (0.0s total), output rate 404MB/s
// INFO: [JOB 1322] compacting: sstable created 001146
// INFO: [JOB 1330] compacting(move) multilevelL3 [001140] (707B) Score=889.58 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=9.55; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1330] compacted(move) multilevelL3 [001140] (707B) Score=889.58 + L4 [] (0B) Score=0.00 + L5 [] (0B) Score=9.55 -> L5 [001140] (707B), in 0.0s (0.0s total), output rate 43MB/s
// INFO: [JOB 1322] compacted(rewrite) L0 [001141] (876B) Score=0.00 + L0 [] (0B) Score=0.00 -> L0 [001146] (711B), in 0.0s (0.0s total), output rate 66KB/s
// INFO: [JOB 1332] compacting(move) multilevelL2 [001142] (782B) Score=5805.00 + L3 [] (0B) Score=0.00 + L4 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 0.00
// INFO: [JOB 1332] compacted(move) multilevelL2 [001142] (782B) Score=5805.00 + L3 [] (0B) Score=0.00 + L4 [] (0B) Score=0.00 -> L4 [001142] (782B), in 0.0s (0.0s total), output rate 227MB/s
// INFO: [JOB 1322] sstable deleted 001141
// INFO: [JOB 1333] compacting(default) L5 [001136 001137 001138 001139 001140] (5.3KB) Score=10.83 + L6 [001131] (3.1KB) Score=0.30; OverlappingRatio: Single 0.59, Multi 0.00
// INFO: [JOB 1333] compacting: sstable created 001147
// INFO: [JOB 1333] compacted(default) L5 [001136 001137 001138 001139 001140] (5.3KB) Score=10.83 + L6 [001131] (3.1KB) Score=0.30 -> L6 [001147] (4.8KB), in 0.0s (0.0s total), output rate 1.5MB/s
// INFO: [JOB 1333] sstable deleted 001131
// INFO: [JOB 1333] sstable deleted 001136
// INFO: [JOB 1333] sstable deleted 001137
// INFO: [JOB 1333] sstable deleted 001138
// INFO: [JOB 1333] sstable deleted 001139
// INFO: [JOB 1333] sstable deleted 001140
// INFO: [JOB 1335] compacting(move) L4 [001142] (782B) Score=299.23 + L5 [] (0B) Score=0.00; OverlappingRatio: Single 0.00, Multi 6.29
// INFO: [JOB 1335] compacted(move) L4 [001142] (782B) Score=299.23 + L5 [] (0B) Score=0.00 -> L5 [001142] (782B), in 0.0s (0.0s total), output rate 170MB/s
To reduce: go test ./internal/metamorphic -tags invariants -run 'TestMeta$' --run-dir /artifacts/meta/240328-041418.8441452601248/random-013 --try-to-reduce -v
--- FAIL: TestMeta/execution/random-013 (62.32s)
Help
To reproduce, try:
go test -tags 'invariants' -exec 'stress -p 1' -timeout 0 -test.v -run 'TestMeta$' ./internal/metamorphic -seed 1711599258844481948 -ops "uniform:5000-10000"github.com/cockroachdb/pebble/internal/metamorphic.TestMeta failed with artifacts on refs/heads/master @ b6e563f67d90:
// INFO: [JOB 119] flushing: sstable created 000236
// INFO: [JOB 111] MANIFEST created 000230
// INFO: [JOB 111] compacted(default) L0 [000110 000113 000101 000112 000111 000105 000126 000130 000127 000128 000129 000119 000118 000165 000139 000161 000142 000160 000159 000157 000162 000144 000148 000158 000164 000163 000177 000178 000179 000180 000181 000175 000176 000167 000182 000183 000189 000190 000191 000192 000193 000194 000188 000184 000195 000205 000207 000209 000206 000208 000211] (34KB) Score=100.00 + L6 [000143 000149 000156 000141] (3.4KB) Score=0.00 -> L6 [000226 000227] (2.2KB), in 0.4s (0.5s total), output rate 5.6KB/s
// INFO: [JOB 111] sstable deleted 000110
// INFO: [JOB 111] sstable deleted 000111
// INFO: [JOB 111] sstable deleted 000112
// INFO: [JOB 111] sstable deleted 000113
// INFO: [JOB 111] sstable deleted 000126
// INFO: [JOB 111] sstable deleted 000127
// INFO: [JOB 111] sstable deleted 000128
// INFO: [JOB 111] sstable deleted 000129
// INFO: [JOB 111] sstable deleted 000130
// INFO: [JOB 111] sstable deleted 000141
// INFO: [JOB 111] sstable deleted 000143
// INFO: [JOB 111] sstable deleted 000149
// INFO: [JOB 111] sstable deleted 000156
// INFO: [JOB 111] sstable deleted 000157
// INFO: [JOB 111] sstable deleted 000159
// INFO: [JOB 111] sstable deleted 000160
// INFO: [JOB 111] sstable deleted 000161
// INFO: [JOB 111] sstable deleted 000162
// INFO: [JOB 111] sstable deleted 000165
// INFO: [JOB 111] sstable deleted 000175
// INFO: [JOB 111] sstable deleted 000176
// INFO: [JOB 111] sstable deleted 000177
// INFO: [JOB 111] sstable deleted 000178
// INFO: [JOB 111] sstable deleted 000179
// INFO: [JOB 111] sstable deleted 000180
// INFO: [JOB 111] sstable deleted 000181
// INFO: [JOB 111] sstable deleted 000188
// INFO: [JOB 111] sstable deleted 000189
// INFO: [JOB 111] sstable deleted 000190
// INFO: [JOB 111] sstable deleted 000191
// INFO: [JOB 111] sstable deleted 000192
// INFO: [JOB 111] sstable deleted 000193
// INFO: [JOB 111] sstable deleted 000194
// INFO: [JOB 111] sstable deleted 000205
// INFO: [JOB 111] sstable deleted 000206
// INFO: [JOB 111] sstable deleted 000207
// INFO: [JOB 111] sstable deleted 000208
// INFO: [JOB 111] sstable deleted 000209
// INFO: [JOB 111] MANIFEST deleted 000166
// INFO: [JOB 119] MANIFEST created 000237
// INFO: [JOB 119] flushed 1 memtable (329B) to L0 [000232 000233 000234 000235 000236] (3.7KB), in 0.1s (0.1s total), output rate 46KB/s
// INFO: [JOB 119] WAL deleted 000213
// INFO: [JOB 119] WAL deleted 000213
// INFO: [JOB 119] MANIFEST deleted 000210
To reduce: go test ./internal/metamorphic -tags invariants -run 'TestMeta$' --run-dir /artifacts/meta/240328-045355.365770946795/random-011 --try-to-reduce -v
--- FAIL: TestMeta/execution/random-011 (305.05s)
Help
To reproduce, try:
go test -tags 'invariants' -exec 'stress -p 1' -timeout 0 -test.v -run 'TestMeta$' ./internal/metamorphic -seed 1711601635365148955 -ops "uniform:5000-10000"
First failure:
panic: operation took longer than 1m0s: db1.Download("lgkpd@15" /* start */, "pufevbbtzg@8" /* end */, false /* viaBackingFileDownload */, "cpkcrjlgbvho@6" /* start */, "zbiwrp@16" /* end */, true /* viaBackingFileDownload */, "lpcnlj@8" /* start */, "zjababk@79" /* end */, false /* viaBackingFileDownload */)
Second failure:
panic: operation took longer than 5m0s: db1.Download("mbyau" /* start */, "qsmnyn@9" /* end */, false /* viaBackingFileDownload */, "gjpn" /* start */, "imvr" /* end */, true /* viaBackingFileDownload */)
Both are #3470 .
All known MetaTest issues have been fixed.