meta: TestOptionsRoundtrip is flaky
sumeerbhola opened this issue · comments
I suspect what is happening below is that the value of opts.ioLatencyProbability is very close to 0, but not zero, so the initial write has these values written:
Lines 240 to 244 in c109764
But since the default precision for printf of floats is causing it to print as zero, the parsing is seeing it as 0, so when the parsed options are printed, we omit these values.
--- FAIL: TestOptionsRoundtrip (0.68s)
--- FAIL: TestOptionsRoundtrip/random-065 (0.00s)
options_test.go:94: Serialized options:
[Version]
pebble_version=0.1
[Options]
bytes_per_sync=65536
cache_size=33554432
cleaner=archive
compaction_debt_concurrency=1073741824
comparer=pebble.internal.testkeys
disable_wal=false
disable_ingest_as_flushable=true
flush_delay_delete_range=905ms
flush_delay_range_key=670ms
flush_split_bytes=128
format_major_version=17
l0_compaction_concurrency=1
l0_compaction_file_threshold=1
l0_compaction_threshold=9
l0_stop_writes_threshold=84
lbase_max_bytes=16
level_multiplier=80
max_concurrent_compactions=1
max_manifest_file_size=1
max_open_files=1000
mem_table_size=4194304
mem_table_stop_writes_threshold=4
min_deletion_rate=16777216
merger=pebble.concatenate
multilevel_compaction_heuristic=wamp(0.00, false)
read_compaction_rate=16000
read_sampling_multiplier=16
strict_wal_tail=true
table_cache_shards=4
validate_on_ingest=false
wal_dir=
wal_bytes_per_sync=0
max_writer_concurrency=0
force_writer_parallelism=false
secondary_cache_size_bytes=0
create_on_shared=0
[WAL Failover]
secondary_dir=data/wal_secondary
primary_dir_probe_interval=1.951817ms
healthy_probe_latency_threshold=4.428586ms
healthy_interval=8.303307ms
unhealthy_sampling_interval=2.092938ms
unhealthy_operation_latency_threshold=6.05706ms
elevated_write_stall_threshold_lag=15.792989ms
[Level "0"]
block_restart_interval=64
block_size=16384
block_size_threshold=90
compression=ZSTD
filter_policy=testing_bloom_filter/bits_per_key=8
filter_type=table
index_block_size=8192
target_file_size=1048576
[TestOptions]
strictfs=true
replace_single_delete=true
threads=4
enable_value_blocks=true
disable_value_blocks_for_ingest_sstables=true
external_storage_enabled=true
seed_efos=5909666[56](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:57)1378333442
io_latency_mean=3.836181ms
io_latency_probability=0.000000
io_latency_seed=4038267658547851335
use_excise=true
options_test.go:100:
Error Trace: /home/runner/work/pebble/pebble/metamorphic/options_test.go:100
/home/runner/work/pebble/pebble/metamorphic/options_test.go:148
Error: Not equal:
expected: "[Version]\n pebble_version=0.1\n\n[Options]\n bytes_per_sync=65536\n cache_size=33554432\n cleaner=archive\n compaction_debt_concurrency=1073741824\n comparer=pebble.internal.testkeys\n disable_wal=false\n disable_ingest_as_flushable=true\n flush_delay_delete_range=905ms\n flush_delay_range_key=670ms\n flush_split_bytes=128\n format_major_version=17\n l0_compaction_concurrency=1\n l0_compaction_file_threshold=1\n l0_compaction_threshold=9\n l0_stop_writes_threshold=84\n lbase_max_bytes=16\n level_multiplier=80\n max_concurrent_compactions=1\n max_manifest_file_size=1\n max_open_files=1000\n mem_table_size=4194304\n mem_table_stop_writes_threshold=4\n min_deletion_rate=16777216\n merger=pebble.concatenate\n multilevel_compaction_heuristic=wamp(0.00, false)\n read_compaction_rate=16000\n read_sampling_multiplier=16\n strict_wal_tail=true\n table_cache_shards=4\n validate_on_ingest=false\n wal_dir=\n wal_bytes_per_sync=0\n max_writer_concurrency=0\n force_writer_parallelism=false\n secondary_cache_size_bytes=0\n create_on_shared=0\n\n[WAL Failover]\n secondary_dir=data/wal_secondary\n primary_dir_probe_interval=1.951817ms\n healthy_probe_latency_threshold=4.428586ms\n healthy_interval=8.303307ms\n unhealthy_sampling_interval=2.092938ms\n unhealthy_operation_latency_threshold=6.0[57](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:58)06ms\n elevated_write_stall_threshold_lag=15.792989ms\n\n[Level \"0\"]\n block_restart_interval=64\n block_size=16384\n block_size_threshold=90\n compression=ZSTD\n filter_policy=testing_bloom_filter/bits_per_key=8\n filter_type=table\n index_block_size=8192\n target_file_size=1048576\n\n[TestOptions]\n strictfs=true\n replace_single_delete=true\n threads=4\n enable_value_blocks=true\n disable_value_blocks_for_ingest_sstables=true\n external_storage_enabled=true\n seed_efos=5909666561378333442\n io_latency_mean=3.836181ms\n io_latency_probability=0.000000\n io_latency_seed=40382676[58](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:59)547851335\n use_excise=true\n"
actual : "[Version]\n pebble_version=0.1\n\n[Options]\n bytes_per_sync=65536\n cache_size=33554432\n cleaner=archive\n compaction_debt_concurrency=1073741824\n comparer=pebble.internal.testkeys\n disable_wal=false\n disable_ingest_as_flushable=true\n flush_delay_delete_range=905ms\n flush_delay_range_key=670ms\n flush_split_bytes=128\n format_major_version=17\n l0_compaction_concurrency=1\n l0_compaction_file_threshold=1\n l0_compaction_threshold=9\n l0_stop_writes_threshold=84\n lbase_max_bytes=16\n level_multiplier=80\n max_concurrent_compactions=1\n max_manifest_file_size=1\n max_open_files=1000\n mem_table_size=4194304\n mem_table_stop_writes_threshold=4\n min_deletion_rate=16777216\n merger=pebble.concatenate\n multilevel_compaction_heuristic=wamp(0.00, false)\n read_compaction_rate=16000\n read_sampling_multiplier=16\n strict_wal_tail=true\n table_cache_shards=4\n validate_on_ingest=false\n wal_dir=\n wal_bytes_per_sync=0\n max_writer_concurrency=0\n force_writer_parallelism=false\n secondary_cache_size_bytes=0\n create_on_shared=0\n\n[WAL Failover]\n secondary_dir=data/wal_secondary\n primary_dir_probe_interval=1.951817ms\n healthy_probe_latency_threshold=4.428586ms\n healthy_interval=8.303307ms\n unhealthy_sampling_interval=2.092938ms\n unhealthy_operation_latency_threshold=6.05706ms\n elevated_write_stall_threshold_lag=15.792989ms\n\n[Level \"0\"]\n block_restart_interval=64\n block_size=16384\n block_size_threshold=90\n compression=ZSTD\n filter_policy=testing_bloom_filter/bits_per_key=8\n filter_type=table\n index_block_size=8192\n target_file_size=1048576\n\n[TestOptions]\n strictfs=true\n replace_single_delete=true\n threads=4\n enable_value_blocks=true\n disable_value_blocks_for_ingest_sstables=true\n external_storage_enabled=true\n seed_efos=[59](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:60)096665[61](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:62)378333442\n use_excise=true\n"
Diff:
--- Expected
+++ Actual
@@ -68,5 +68,2 @@
seed_efos=590966[65](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:66)61378333442
- io_latency_mean=3.836181ms
- io_latency_probability=0.000000
- io_latency_seed=40382[67](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:68)65854[78](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:79)5[133](https://github.com/cockroachdb/pebble/actions/runs/8331540259/job/22798668146?pr=3407#step:4:134)5
use_excise=true
Test: TestOptionsRoundtrip/random-065
FAIL