[BUG] Potential Memory leak?
JachymPutta opened this issue · comments
Describe the bug
Might be related to #7787
When stress-testing a redis instance, request increases the memory footprint of the instance. I imagine there is some wait-queue for the incoming requests when the instance doesn't process them fast enough?
To reproduce
Reproducing this bug requires a client to generate the necessary request volume. I used YCSB and a custom client.
- Create a redis instance with
allkeys-lrupolicy or set with (config set maxmemory-policy allkeys-lru) - Fill the instance with keys
- Set a memory limit that restricts the instance (
config set maxmemory <limit>), the higher request rate the client is able to generate the higher memory limit is possible - Check the number of keys present (
dbsize) - Run the client (YCSB or other) to spam the instance with requests (request type doesn't matter)
- The keys should be gone (
dbsize== 0)
Expected behavior
I would expect that sending requests to the instance shouldn't add memory overhead. And evict keys
Additional information
Sometimes the issue wouldn't manifest, I am not sure whether it has anything to do with me checking the info while running the requests.
# Server
redis_version:255.255.255
redis_git_sha1:7b070423
redis_git_dirty:0
redis_build_id:ec3d40a800e4f4e2
redis_mode:standalone
os:Linux 6.5.11-7-pve x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:12.2.0
process_id:1222855
process_supervised:no
run_id:be1fce769a4425c669aa3a9ab963be298470a73d
tcp_port:6379
Memory info before the request spam:
dbsize = 895
# Memory
used_memory:10355320
used_memory_human:9.88M
used_memory_rss:129445888
used_memory_rss_human:123.45M
used_memory_peak:10313513592
used_memory_peak_human:9.61G
used_memory_peak_perc:0.10%
used_memory_overhead:925184
used_memory_startup:874776
used_memory_dataset:9430136
used_memory_dataset_perc:99.47%
allocator_allocated:10614832
allocator_active:19861504
allocator_resident:137224192
allocator_muzzy:0
total_system_memory:270033489920
total_system_memory_human:251.49G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:10485760
maxmemory_human:10.00M
maxmemory_policy:allkeys-lru
allocator_frag_ratio:1.87
allocator_frag_bytes:9246672
allocator_rss_ratio:6.91
allocator_rss_bytes:117362688
rss_overhead_ratio:0.94
rss_overhead_bytes:-7778304
mem_fragmentation_ratio:12.53
mem_fragmentation_bytes:119111240
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:5784
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
mem_overhead_db_hashtable_rehashing:0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
State after running the client
dbsize = 0
# Memory
used_memory:1122656
used_memory_human:1.07M
used_memory_rss:111198208
used_memory_rss_human:106.05M
used_memory_peak:10313513592
used_memory_peak_human:9.61G
used_memory_peak_perc:0.01%
used_memory_overhead:879296
used_memory_startup:874776
used_memory_dataset:243360
used_memory_dataset_perc:98.18%
allocator_allocated:1456728
allocator_active:2179072
allocator_resident:119541760
allocator_muzzy:0
total_system_memory:270033489920
total_system_memory_human:251.49G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:10485760
maxmemory_human:10.00M
maxmemory_policy:allkeys-lru
allocator_frag_ratio:1.50
allocator_frag_bytes:722344
allocator_rss_ratio:54.86
allocator_rss_bytes:117362688
rss_overhead_ratio:0.93
rss_overhead_bytes:-8343552
mem_fragmentation_ratio:100.91
mem_fragmentation_bytes:110096224
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:3856
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
mem_overhead_db_hashtable_rehashing:0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
9:C 31 Mar 2024 13:45:49.233 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 31 Mar 2024 13:45:49.233 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 31 Mar 2024 13:45:49.233 * Configuration loaded
9:M 31 Mar 2024 13:45:49.234 * monotonic clock: POSIX clock_gettime
9:M 31 Mar 2024 13:45:49.234 * Running mode=standalone, port=6379.
9:M 31 Mar 2024 13:45:49.234 * Module 'RedisCompat' loaded from /opt/redis-stack/lib/rediscompat.so
9:M 31 Mar 2024 13:45:49.236 * Redis version found by RedisSearch : 7.2.4 - oss
9:M 31 Mar 2024 13:45:49.236 * RediSearch version 2.8.12 (Git=2.8-32fdaca)
9:M 31 Mar 2024 13:45:49.236 * Low level api version 1 initialized successfully
9:M 31 Mar 2024 13:45:49.236 * concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results: 10000, search pool size: 20, index pool size: 8,
9:M 31 Mar 2024 13:45:49.236 * Initialized thread pools!
9:M 31 Mar 2024 13:45:49.236 * Enabled role change notification
9:M 31 Mar 2024 13:45:49.236 * Module 'search' loaded from /opt/redis-stack/lib/redisearch.so
9:M 31 Mar 2024 13:45:49.237 * RedisTimeSeries version 11011, git_sha=0299ac12a6bf298028859c41ba0f4d8dc842726b
9:M 31 Mar 2024 13:45:49.237 * Redis version found by RedisTimeSeries : 7.2.4 - oss
9:M 31 Mar 2024 13:45:49.237 * loaded default CHUNK_SIZE_BYTES policy: 4096
9:M 31 Mar 2024 13:45:49.237 * loaded server DUPLICATE_POLICY: block
9:M 31 Mar 2024 13:45:49.237 * Setting default series ENCODING to: compressed
9:M 31 Mar 2024 13:45:49.237 * Detected redis oss
9:M 31 Mar 2024 13:45:49.237 * Module 'timeseries' loaded from /opt/redis-stack/lib/redistimeseries.so
9:M 31 Mar 2024 13:45:49.238 * Created new data type 'ReJSON-RL'
9:M 31 Mar 2024 13:45:49.238 * version: 20609 git sha: unknown branch: unknown
9:M 31 Mar 2024 13:45:49.238 * Exported RedisJSON_V1 API
9:M 31 Mar 2024 13:45:49.238 * Exported RedisJSON_V2 API
9:M 31 Mar 2024 13:45:49.238 * Exported RedisJSON_V3 API
9:M 31 Mar 2024 13:45:49.238 * Exported RedisJSON_V4 API
9:M 31 Mar 2024 13:45:49.238 * Exported RedisJSON_V5 API
9:M 31 Mar 2024 13:45:49.238 * Enabled diskless replication
9:M 31 Mar 2024 13:45:49.238 * Module 'ReJSON' loaded from /opt/redis-stack/lib/rejson.so
9:M 31 Mar 2024 13:45:49.238 * Acquired RedisJSON_V5 API
9:M 31 Mar 2024 13:45:49.238 * RedisBloom version 2.6.12 (Git=unknown)
9:M 31 Mar 2024 13:45:49.238 * Module 'bf' loaded from /opt/redis-stack/lib/redisbloom.so
9:M 31 Mar 2024 13:45:49.238 * <redisgears_2> Created new data type 'GearsType'
9:M 31 Mar 2024 13:45:49.239 * <redisgears_2> Detected redis oss
9:M 31 Mar 2024 13:45:49.239 # <redisgears_2> could not initialize RedisAI_InitError
9:M 31 Mar 2024 13:45:49.239 * <redisgears_2> Failed loading RedisAI API.
9:M 31 Mar 2024 13:45:49.239 * <redisgears_2> RedisGears v2.0.19, sha='671030bbcb7de4582d00575a0902f826da3efe73', build_type='release', built_for='Linux-ubuntu22.04.x86_64'.
=== REDIS BUG REPORT START: Cut & paste starting from here ===
9:M 31 Mar 2024 13:45:49.239 # Redis 7.2.4 crashed by signal: 7, si_code: 2
9:M 31 Mar 2024 13:45:49.239 # Accessing address: 0x7f84c271a7d8
9:M 31 Mar 2024 13:45:49.239 # Crashed running the instruction at: 0x7f84c5cd31da
------ STACK TRACE ------
EIP:
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7f84c5cd31da]
Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f84c54ee520]
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7f84c5cd31da]
/lib64/ld-linux-x86-64.so.2(+0x902c)[0x7f84c5cb102c]
/lib64/ld-linux-x86-64.so.2(+0xa601)[0x7f84c5cb2601]
/lib64/ld-linux-x86-64.so.2(+0xe9a9)[0x7f84c5cb69a9]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f84c5620a98]
/lib64/ld-linux-x86-64.so.2(+0xdf9a)[0x7f84c5cb5f9a]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f84c5620a98]
/lib64/ld-linux-x86-64.so.2(+0xe34e)[0x7f84c5cb634e]
/lib/x86_64-linux-gnu/libc.so.6(+0x9063c)[0x7f84c553c63c]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f84c5620a98]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x33)[0x7f84c5620b63]
/lib/x86_64-linux-gnu/libc.so.6(+0x9012e)[0x7f84c553c12e]
/lib/x86_64-linux-gnu/libc.so.6(dlopen+0x48)[0x7f84c553c6c8]
/opt/redis-stack/lib/redisgears.so(+0xccb11)[0x7f84c2b97b11]
/opt/redis-stack/lib/redisgears.so(+0x73dd8)[0x7f84c2b3edd8]
/opt/redis-stack/lib/redisgears.so(RedisModule_OnLoad+0x1610)[0x7f84c2b7c530]
/opt/redis-stack/bin/redis-server *:6379(moduleLoad+0xab)[0x55af781e7aeb]
/opt/redis-stack/bin/redis-server *:6379(moduleLoadFromQueue+0x4a)[0x55af781e7e9a]
/opt/redis-stack/bin/redis-server *:6379(main+0x7ed)[0x55af780d7a4d]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f84c54d5d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f84c54d5e40]
/opt/redis-stack/bin/redis-server *:6379(_start+0x25)[0x55af780d7e05]
------ REGISTERS ------
9:M 31 Mar 2024 13:45:49.239 #
RAX:00007f84c271a7d8 RBX:00007ffd69c398f0
RCX:00007f84c273f230 RDX:0000000000000828
RDI:00007f84c271a7d8 RSI:0000000000000000
RBP:00007ffd69c39c80 RSP:00007ffd69c398e8
R8 :00007f84c271a7d8 R9 :000000000193c000
R10:0000000000000812 R11:0000000000000003
R12:0000000000000004 R13:00007ffd69c39998
R14:00007f84c271b000 R15:000055af7972e7d0
RIP:00007f84c5cd31da EFL:0000000000010202
CSGSFS:002b000000000033
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f7) -> 00000000014cc000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f6) -> 00007f8400000005
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f5) -> 0000000000110000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f4) -> 0000000000001000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f3) -> 00000000014cb049
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f2) -> 00000000014cb049
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f1) -> 00000000014cc000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398f0) -> 0000000000110000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398ef) -> 0000000000000001
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398ee) -> 0000000000000000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398ed) -> 0000000000001000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398ec) -> 000000000010f588
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398eb) -> 000000000010f588
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398ea) -> 0000000000110000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398e9) -> 0000000000000000
9:M 31 Mar 2024 13:45:49.239 # (00007ffd69c398e8) -> 00007f84c5cb102c
------ INFO OUTPUT ------
Server
redis_version:7.2.4
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:3a23a5ac67fe7008
redis_mode:standalone
os:Linux 5.15.133.1-microsoft-standard-WSL2 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:11.4.0
process_id:9
process_supervised:no
run_id:8eb38d7c7a59ffbe10fb5e2ce9732351f3e46f97
tcp_port:6379
server_time_usec:1711892749238858
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:616717
executable:/opt/redis-stack/bin/redis-server
config_file:
io_threads_active:0
Clients
connected_clients:0
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
Memory
used_memory:1170184
used_memory_human:1.12M
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:1170184
used_memory_peak_human:1.12M
used_memory_peak_perc:100.81%
used_memory_overhead:184
used_memory_startup:0
used_memory_dataset:1170000
used_memory_dataset_perc:99.98%
allocator_allocated:0
allocator_active:0
allocator_resident:0
total_system_memory:4004585472
total_system_memory_human:3.73G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:-nan
allocator_frag_bytes:0
allocator_rss_ratio:-nan
allocator_rss_bytes:0
rss_overhead_ratio:-nan
rss_overhead_bytes:0
mem_fragmentation_ratio:-nan
mem_fragmentation_bytes:0
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1711892749
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
Stats
total_connections_received:0
total_commands_processed:5
instantaneous_ops_per_sec:0
total_net_input_bytes:0
total_net_output_bytes:0
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:0
total_writes_processed:0
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
eventloop_cycles:0
eventloop_duration_sum:0
eventloop_duration_cmd_sum:0
instantaneous_eventloop_cycles_per_sec:0
instantaneous_eventloop_duration_usec:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:482663b458a6ffc71d3854da8a46af0e6e29a868
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
CPU
used_cpu_sys:0.000000
used_cpu_user:0.009057
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.008953
Modules
module:name=ReJSON,ver=20609,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors]
module:name=timeseries,ver=11011,api=1,filters=0,usedby=[],using=[],options=[]
module:name=search,ver=20812,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
module:name=RedisCompat,ver=1,api=1,filters=0,usedby=[],using=[],options=[]
module:name=bf,ver=20612,api=1,filters=0,usedby=[],using=[],options=[]
Commandstats
cmdstat_info:calls=5,usec=46,usec_per_call=9.20,rejected_calls=0,failed_calls=0
Errorstats
Latencystats
latency_percentiles_usec_info:p50=8.031,p99=22.015,p99.9=22.015
Cluster
cluster_enabled:0
Keyspace
------ CLIENT LIST OUTPUT ------
------ MODULES INFO OUTPUT ------
ReJSON_trace
ReJSON_backtrace: 0: redis_module::basic_info_command_handler
1: rejson::__info_func
2: modulesCollectInfo
at /__w/redis-stack/redis-stack/redis/src/module.c:10293:9
3: logModulesInfo
at /__w/redis-stack/redis-stack/redis/src/debug.c:1900:22
printCrashReport
at /__w/redis-stack/redis-stack/redis/src/debug.c:2183:5
4: sigsegvHandler
at /__w/redis-stack/redis-stack/redis/src/debug.c:2164:5
5:
6:
7:
8:
9:
10: _dl_catch_exception
11:
12: _dl_catch_exception
13:
14:
15: _dl_catch_exception
16: _dl_catch_error
17:
18: dlopen
19: libloading::os::unix::Library::open
20: redisgears::js_init
21: RedisModule_OnLoad
22: moduleLoad
at /__w/redis-stack/redis-stack/redis/src/module.c:12161:9
23: moduleLoadFromQueue
at /__w/redis-stack/redis-stack/redis/src/module.c:11956:13
24: main
at /__w/redis-stack/redis-stack/redis/src/server.c:7308:9
25:
26: __libc_start_main
27: _start
search_version
search_version:2.8.12
search_redis_version:7.2.4 - oss
search_index
search_number_of_indexes:0
search_fields_statistics
search_dialect_statistics
search_dialect_1:0
search_dialect_2:0
search_dialect_3:0
search_dialect_4:0
search_runtime_configurations
search_concurrent_mode:OFF
search_enableGC:ON
search_minimal_term_prefix:2
search_maximal_prefix_expansions:200
search_query_timeout_ms:500
search_timeout_policy:return
search_cursor_read_size:1000
search_cursor_max_idle_time:300000
search_max_doc_table_size:1000000
search_max_search_results:10000
search_max_aggregate_results:10000
search_search_pool_size:20
search_index_pool_size:8
search_gc_scan_size:100
search_min_phonetic_term_length:3
------ CONFIG DEBUG OUTPUT ------
replica-read-only yes
proto-max-bulk-len 512mb
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
slave-read-only yes
list-compress-depth 0
lazyfree-lazy-server-del no
lazyfree-lazy-user-flush no
activedefrag no
io-threads 1
repl-diskless-load disabled
repl-diskless-sync yes
io-threads-do-reads no
lazyfree-lazy-user-del no
sanitize-dump-payload no
client-query-buffer-limit 1gb
------ FAST MEMORY TEST ------
*** Preparing to test memory region 55af783d7000 (704512 bytes)
*** Preparing to test memory region 55af796df000 (450560 bytes)
*** Preparing to test memory region 7f84bf0b0000 (9437184 bytes)
*** Preparing to test memory region 7f84c2740000 (3670016 bytes)
*** Preparing to test memory region 7f84c2e60000 (8192 bytes)
*** Preparing to test memory region 7f84c33c5000 (12288 bytes)
*** Preparing to test memory region 7f84c345e000 (4096 bytes)
*** Preparing to test memory region 7f84c3460000 (8388608 bytes)
*** Preparing to test memory region 7f84c3c61000 (8388608 bytes)
*** Preparing to test memory region 7f84c4a79000 (28672 bytes)
*** Preparing to test memory region 7f84c4a80000 (9961472 bytes)
*** Preparing to test memory region 7f84c54a7000 (20480 bytes)
*** Preparing to test memory region 7f84c56c8000 (53248 bytes)
*** Preparing to test memory region 7f84c5b16000 (12288 bytes)
*** Preparing to test memory region 7f84c5ca6000 (8192 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: (null) (base: (nil))
Module: /lib64/ld-linux-x86-64.so.2 (base 0x7f84c5ca8000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/redis/redis/issues
If a Redis module was involved, please open in the module's repo instead.
Suspect RAM error? Use redis-server --test-memory to verify it.
Some other issues could be detected by redis-server --check-system
Bus error
9:C 31 Mar 2024 13:46:08.436 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 31 Mar 2024 13:46:08.436 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 31 Mar 2024 13:46:08.436 * Configuration loaded
9:M 31 Mar 2024 13:46:08.436 * monotonic clock: POSIX clock_gettime
9:M 31 Mar 2024 13:46:08.437 * Running mode=standalone, port=6379.
9:M 31 Mar 2024 13:46:08.437 * Module 'RedisCompat' loaded from /opt/redis-stack/lib/rediscompat.so
9:M 31 Mar 2024 13:46:08.439 * Redis version found by RedisSearch : 7.2.4 - oss
9:M 31 Mar 2024 13:46:08.439 * RediSearch version 2.8.12 (Git=2.8-32fdaca)
9:M 31 Mar 2024 13:46:08.439 * Low level api version 1 initialized successfully
9:M 31 Mar 2024 13:46:08.439 * concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results: 10000, search pool size: 20, index pool size: 8,
9:M 31 Mar 2024 13:46:08.439 * Initialized thread pools!
9:M 31 Mar 2024 13:46:08.439 * Enabled role change notification
9:M 31 Mar 2024 13:46:08.440 * Module 'search' loaded from /opt/redis-stack/lib/redisearch.so
9:M 31 Mar 2024 13:46:08.441 * RedisTimeSeries version 11011, git_sha=0299ac12a6bf298028859c41ba0f4d8dc842726b
9:M 31 Mar 2024 13:46:08.441 * Redis version found by RedisTimeSeries : 7.2.4 - oss
9:M 31 Mar 2024 13:46:08.441 * loaded default CHUNK_SIZE_BYTES policy: 4096
9:M 31 Mar 2024 13:46:08.441 * loaded server DUPLICATE_POLICY: block
9:M 31 Mar 2024 13:46:08.441 * Setting default series ENCODING to: compressed
9:M 31 Mar 2024 13:46:08.441 * Detected redis oss
9:M 31 Mar 2024 13:46:08.441 * Module 'timeseries' loaded from /opt/redis-stack/lib/redistimeseries.so
9:M 31 Mar 2024 13:46:08.441 * Created new data type 'ReJSON-RL'
9:M 31 Mar 2024 13:46:08.442 * version: 20609 git sha: unknown branch: unknown
9:M 31 Mar 2024 13:46:08.442 * Exported RedisJSON_V1 API
9:M 31 Mar 2024 13:46:08.442 * Exported RedisJSON_V2 API
9:M 31 Mar 2024 13:46:08.442 * Exported RedisJSON_V3 API
9:M 31 Mar 2024 13:46:08.442 * Exported RedisJSON_V4 API
9:M 31 Mar 2024 13:46:08.442 * Exported RedisJSON_V5 API
9:M 31 Mar 2024 13:46:08.442 * Enabled diskless replication
9:M 31 Mar 2024 13:46:08.442 * Module 'ReJSON' loaded from /opt/redis-stack/lib/rejson.so
9:M 31 Mar 2024 13:46:08.442 * Acquired RedisJSON_V5 API
9:M 31 Mar 2024 13:46:08.442 * RedisBloom version 2.6.12 (Git=unknown)
9:M 31 Mar 2024 13:46:08.442 * Module 'bf' loaded from /opt/redis-stack/lib/redisbloom.so
9:M 31 Mar 2024 13:46:08.452 * <redisgears_2> Created new data type 'GearsType'
9:M 31 Mar 2024 13:46:08.452 * <redisgears_2> Detected redis oss
9:M 31 Mar 2024 13:46:08.452 # <redisgears_2> could not initialize RedisAI_InitError
9:M 31 Mar 2024 13:46:08.452 * <redisgears_2> Failed loading RedisAI API.
9:M 31 Mar 2024 13:46:08.452 * <redisgears_2> RedisGears v2.0.19, sha='671030bbcb7de4582d00575a0902f826da3efe73', build_type='release', built_for='Linux-ubuntu22.04.x86_64'.
=== REDIS BUG REPORT START: Cut & paste starting from here ===
9:M 31 Mar 2024 13:46:08.453 # Redis 7.2.4 crashed by signal: 7, si_code: 2
9:M 31 Mar 2024 13:46:08.453 # Accessing address: 0x7fbcc8afa7d8
9:M 31 Mar 2024 13:46:08.453 # Crashed running the instruction at: 0x7fbccc0691da
------ STACK TRACE ------
EIP:
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7fbccc0691da]
Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fbccb884520]
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7fbccc0691da]
/lib64/ld-linux-x86-64.so.2(+0x902c)[0x7fbccc04702c]
/lib64/ld-linux-x86-64.so.2(+0xa601)[0x7fbccc048601]
/lib64/ld-linux-x86-64.so.2(+0xe9a9)[0x7fbccc04c9a9]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7fbccb9b6a98]
/lib64/ld-linux-x86-64.so.2(+0xdf9a)[0x7fbccc04bf9a]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7fbccb9b6a98]
/lib64/ld-linux-x86-64.so.2(+0xe34e)[0x7fbccc04c34e]
/lib/x86_64-linux-gnu/libc.so.6(+0x9063c)[0x7fbccb8d263c]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7fbccb9b6a98]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x33)[0x7fbccb9b6b63]
/lib/x86_64-linux-gnu/libc.so.6(+0x9012e)[0x7fbccb8d212e]
/lib/x86_64-linux-gnu/libc.so.6(dlopen+0x48)[0x7fbccb8d26c8]
/opt/redis-stack/lib/redisgears.so(+0xccb11)[0x7fbcc8f71b11]
/opt/redis-stack/lib/redisgears.so(+0x73dd8)[0x7fbcc8f18dd8]
/opt/redis-stack/lib/redisgears.so(RedisModule_OnLoad+0x1610)[0x7fbcc8f56530]
/opt/redis-stack/bin/redis-server *:6379(moduleLoad+0xab)[0x561934c2faeb]
/opt/redis-stack/bin/redis-server *:6379(moduleLoadFromQueue+0x4a)[0x561934c2fe9a]
/opt/redis-stack/bin/redis-server *:6379(main+0x7ed)[0x561934b1fa4d]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fbccb86bd90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fbccb86be40]
/opt/redis-stack/bin/redis-server *:6379(_start+0x25)[0x561934b1fe05]
------ REGISTERS ------
9:M 31 Mar 2024 13:46:08.453 #
RAX:00007fbcc8afa7d8 RBX:00007fffb3eddc90
RCX:00007fbcc8b1f230 RDX:0000000000000828
RDI:00007fbcc8afa7d8 RSI:0000000000000000
RBP:00007fffb3ede020 RSP:00007fffb3eddc88
R8 :00007fbcc8afa7d8 R9 :000000000193c000
R10:0000000000000812 R11:0000000000000003
R12:0000000000000004 R13:00007fffb3eddd38
R14:00007fbcc8afb000 R15:000056193599a7d0
RIP:00007fbccc0691da EFL:0000000000010202
CSGSFS:002b000000000033
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc97) -> 00000000014cc000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc96) -> 00007fbc00000005
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc95) -> 0000000000110000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc94) -> 0000000000001000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc93) -> 00000000014cb049
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc92) -> 00000000014cb049
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc91) -> 00000000014cc000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc90) -> 0000000000110000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8f) -> 0000000000000001
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8e) -> 0000000000000000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8d) -> 0000000000001000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8c) -> 000000000010f588
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8b) -> 000000000010f588
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc8a) -> 0000000000110000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc89) -> 0000000000000000
9:M 31 Mar 2024 13:46:08.453 # (00007fffb3eddc88) -> 00007fbccc04702c
------ INFO OUTPUT ------
Server
redis_version:7.2.4
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:3a23a5ac67fe7008
redis_mode:standalone
os:Linux 5.15.133.1-microsoft-standard-WSL2 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:11.4.0
process_id:9
process_supervised:no
run_id:9c8905d4b5fb4344992bfa1b19a3c2c41750c2c7
tcp_port:6379
server_time_usec:1711892768452539
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:616736
executable:/opt/redis-stack/bin/redis-server
config_file:
io_threads_active:0
Clients
connected_clients:0
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
Memory
used_memory:1170024
used_memory_human:1.12M
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:1170024
used_memory_peak_human:1.12M
used_memory_peak_perc:100.81%
used_memory_overhead:184
used_memory_startup:0
used_memory_dataset:1169840
used_memory_dataset_perc:99.98%
allocator_allocated:0
allocator_active:0
allocator_resident:0
total_system_memory:4004585472
total_system_memory_human:3.73G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:-nan
allocator_frag_bytes:0
allocator_rss_ratio:-nan
allocator_rss_bytes:0
rss_overhead_ratio:-nan
rss_overhead_bytes:0
mem_fragmentation_ratio:-nan
mem_fragmentation_bytes:0
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1711892768
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
Stats
total_connections_received:0
total_commands_processed:5
instantaneous_ops_per_sec:0
total_net_input_bytes:0
total_net_output_bytes:0
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:0
total_writes_processed:0
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
eventloop_cycles:0
eventloop_duration_sum:0
eventloop_duration_cmd_sum:0
instantaneous_eventloop_cycles_per_sec:0
instantaneous_eventloop_duration_usec:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:dee55bc9d4f2881688ffd76607523ac0c6949540
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
CPU
used_cpu_sys:0.014650
used_cpu_user:0.007325
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.014544
used_cpu_user_main_thread:0.007272
Modules
module:name=timeseries,ver=11011,api=1,filters=0,usedby=[],using=[],options=[]
module:name=RedisCompat,ver=1,api=1,filters=0,usedby=[],using=[],options=[]
module:name=ReJSON,ver=20609,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors]
module:name=search,ver=20812,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
module:name=bf,ver=20612,api=1,filters=0,usedby=[],using=[],options=[]
Commandstats
cmdstat_info:calls=5,usec=68,usec_per_call=13.60,rejected_calls=0,failed_calls=0
Errorstats
Latencystats
latency_percentiles_usec_info:p50=12.031,p99=34.047,p99.9=34.047
Cluster
cluster_enabled:0
Keyspace
------ CLIENT LIST OUTPUT ------
------ MODULES INFO OUTPUT ------
ReJSON_trace
ReJSON_backtrace: 0: redis_module::basic_info_command_handler
1: rejson::__info_func
2: modulesCollectInfo
at /__w/redis-stack/redis-stack/redis/src/module.c:10293:9
3: logModulesInfo
at /__w/redis-stack/redis-stack/redis/src/debug.c:1900:22
printCrashReport
at /__w/redis-stack/redis-stack/redis/src/debug.c:2183:5
4: sigsegvHandler
at /__w/redis-stack/redis-stack/redis/src/debug.c:2164:5
5:
6:
7:
8:
9:
10: _dl_catch_exception
11:
12: _dl_catch_exception
13:
14:
15: _dl_catch_exception
16: _dl_catch_error
17:
18: dlopen
19: libloading::os::unix::Library::open
20: redisgears::js_init
21: RedisModule_OnLoad
22: moduleLoad
at /__w/redis-stack/redis-stack/redis/src/module.c:12161:9
23: moduleLoadFromQueue
at /__w/redis-stack/redis-stack/redis/src/module.c:11956:13
24: main
at /__w/redis-stack/redis-stack/redis/src/server.c:7308:9
25:
26: __libc_start_main
27: _start
search_version
search_version:2.8.12
search_redis_version:7.2.4 - oss
search_index
search_number_of_indexes:0
search_fields_statistics
search_dialect_statistics
search_dialect_1:0
search_dialect_2:0
search_dialect_3:0
search_dialect_4:0
search_runtime_configurations
search_concurrent_mode:OFF
search_enableGC:ON
search_minimal_term_prefix:2
search_maximal_prefix_expansions:200
search_query_timeout_ms:500
search_timeout_policy:return
search_cursor_read_size:1000
search_cursor_max_idle_time:300000
search_max_doc_table_size:1000000
search_max_search_results:10000
search_max_aggregate_results:10000
search_search_pool_size:20
search_index_pool_size:8
search_gc_scan_size:100
search_min_phonetic_term_length:3
------ CONFIG DEBUG OUTPUT ------
repl-diskless-load disabled
client-query-buffer-limit 1gb
activedefrag no
io-threads-do-reads no
sanitize-dump-payload no
slave-read-only yes
lazyfree-lazy-expire no
lazyfree-lazy-eviction no
proto-max-bulk-len 512mb
replica-read-only yes
lazyfree-lazy-user-del no
lazyfree-lazy-user-flush no
list-compress-depth 0
io-threads 1
lazyfree-lazy-server-del no
repl-diskless-sync yes
------ FAST MEMORY TEST ------
*** Preparing to test memory region 561934e1f000 (704512 bytes)
*** Preparing to test memory region 56193594b000 (450560 bytes)
*** Preparing to test memory region 7fbcc5490000 (9437184 bytes)
*** Preparing to test memory region 7fbcc8b20000 (3670016 bytes)
*** Preparing to test memory region 7fbcc923a000 (8192 bytes)
*** Preparing to test memory region 7fbcc97c5000 (12288 bytes)
*** Preparing to test memory region 7fbcc985e000 (4096 bytes)
*** Preparing to test memory region 7fbcc9860000 (8388608 bytes)
*** Preparing to test memory region 7fbcca061000 (8388608 bytes)
*** Preparing to test memory region 7fbccae79000 (28672 bytes)
*** Preparing to test memory region 7fbccae80000 (9961472 bytes)
*** Preparing to test memory region 7fbccb83d000 (20480 bytes)
*** Preparing to test memory region 7fbccba5e000 (53248 bytes)
*** Preparing to test memory region 7fbccbeac000 (12288 bytes)
*** Preparing to test memory region 7fbccc03c000 (8192 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: (null) (base: (nil))
Module: /lib64/ld-linux-x86-64.so.2 (base 0x7fbccc03e000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/redis/redis/issues
If a Redis module was involved, please open in the module's repo instead.
Suspect RAM error? Use redis-server --test-memory to verify it.
Some other issues could be detected by redis-server --check-system
Bus error
9:C 31 Mar 2024 13:46:13.459 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 31 Mar 2024 13:46:13.459 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 31 Mar 2024 13:46:13.459 * Configuration loaded
9:M 31 Mar 2024 13:46:13.459 * monotonic clock: POSIX clock_gettime
9:M 31 Mar 2024 13:46:13.460 * Running mode=standalone, port=6379.
9:M 31 Mar 2024 13:46:13.460 * Module 'RedisCompat' loaded from /opt/redis-stack/lib/rediscompat.so
9:M 31 Mar 2024 13:46:13.461 * Redis version found by RedisSearch : 7.2.4 - oss
9:M 31 Mar 2024 13:46:13.461 * RediSearch version 2.8.12 (Git=2.8-32fdaca)
9:M 31 Mar 2024 13:46:13.461 * Low level api version 1 initialized successfully
9:M 31 Mar 2024 13:46:13.461 * concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results: 10000, search pool size: 20, index pool size: 8,
9:M 31 Mar 2024 13:46:13.462 * Initialized thread pools!
9:M 31 Mar 2024 13:46:13.462 * Enabled role change notification
9:M 31 Mar 2024 13:46:13.462 * Module 'search' loaded from /opt/redis-stack/lib/redisearch.so
9:M 31 Mar 2024 13:46:13.463 * RedisTimeSeries version 11011, git_sha=0299ac12a6bf298028859c41ba0f4d8dc842726b
9:M 31 Mar 2024 13:46:13.463 * Redis version found by RedisTimeSeries : 7.2.4 - oss
9:M 31 Mar 2024 13:46:13.463 * loaded default CHUNK_SIZE_BYTES policy: 4096
9:M 31 Mar 2024 13:46:13.463 * loaded server DUPLICATE_POLICY: block
9:M 31 Mar 2024 13:46:13.463 * Setting default series ENCODING to: compressed
9:M 31 Mar 2024 13:46:13.463 * Detected redis oss
9:M 31 Mar 2024 13:46:13.464 * Module 'timeseries' loaded from /opt/redis-stack/lib/redistimeseries.so
9:M 31 Mar 2024 13:46:13.464 * Created new data type 'ReJSON-RL'
9:M 31 Mar 2024 13:46:13.464 * version: 20609 git sha: unknown branch: unknown
9:M 31 Mar 2024 13:46:13.464 * Exported RedisJSON_V1 API
9:M 31 Mar 2024 13:46:13.464 * Exported RedisJSON_V2 API
9:M 31 Mar 2024 13:46:13.464 * Exported RedisJSON_V3 API
9:M 31 Mar 2024 13:46:13.464 * Exported RedisJSON_V4 API
9:M 31 Mar 2024 13:46:13.464 * Exported RedisJSON_V5 API
9:M 31 Mar 2024 13:46:13.464 * Enabled diskless replication
9:M 31 Mar 2024 13:46:13.464 * Module 'ReJSON' loaded from /opt/redis-stack/lib/rejson.so
9:M 31 Mar 2024 13:46:13.464 * Acquired RedisJSON_V5 API
9:M 31 Mar 2024 13:46:13.464 * RedisBloom version 2.6.12 (Git=unknown)
9:M 31 Mar 2024 13:46:13.464 * Module 'bf' loaded from /opt/redis-stack/lib/redisbloom.so
9:M 31 Mar 2024 13:46:13.465 * <redisgears_2> Created new data type 'GearsType'
9:M 31 Mar 2024 13:46:13.465 * <redisgears_2> Detected redis oss
9:M 31 Mar 2024 13:46:13.465 # <redisgears_2> could not initialize RedisAI_InitError
9:M 31 Mar 2024 13:46:13.465 * <redisgears_2> Failed loading RedisAI API.
9:M 31 Mar 2024 13:46:13.465 * <redisgears_2> RedisGears v2.0.19, sha='671030bbcb7de4582d00575a0902f826da3efe73', build_type='release', built_for='Linux-ubuntu22.04.x86_64'.
=== REDIS BUG REPORT START: Cut & paste starting from here ===
9:M 31 Mar 2024 13:46:13.465 # Redis 7.2.4 crashed by signal: 7, si_code: 2
9:M 31 Mar 2024 13:46:13.465 # Accessing address: 0x7f80239ba7d8
9:M 31 Mar 2024 13:46:13.465 # Crashed running the instruction at: 0x7f8026fa51da
------ STACK TRACE ------
EIP:
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7f8026fa51da]
Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f80267c0520]
/lib64/ld-linux-x86-64.so.2(+0x2b1da)[0x7f8026fa51da]
/lib64/ld-linux-x86-64.so.2(+0x902c)[0x7f8026f8302c]
/lib64/ld-linux-x86-64.so.2(+0xa601)[0x7f8026f84601]
/lib64/ld-linux-x86-64.so.2(+0xe9a9)[0x7f8026f889a9]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f80268f2a98]
/lib64/ld-linux-x86-64.so.2(+0xdf9a)[0x7f8026f87f9a]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f80268f2a98]
/lib64/ld-linux-x86-64.so.2(+0xe34e)[0x7f8026f8834e]
/lib/x86_64-linux-gnu/libc.so.6(+0x9063c)[0x7f802680e63c]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88)[0x7f80268f2a98]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x33)[0x7f80268f2b63]
/lib/x86_64-linux-gnu/libc.so.6(+0x9012e)[0x7f802680e12e]
/lib/x86_64-linux-gnu/libc.so.6(dlopen+0x48)[0x7f802680e6c8]
/opt/redis-stack/lib/redisgears.so(+0xccb11)[0x7f8023e2eb11]
/opt/redis-stack/lib/redisgears.so(+0x73dd8)[0x7f8023dd5dd8]
/opt/redis-stack/lib/redisgears.so(RedisModule_OnLoad+0x1610)[0x7f8023e13530]
/opt/redis-stack/bin/redis-server *:6379(moduleLoad+0xab)[0x557e6567faeb]
/opt/redis-stack/bin/redis-server *:6379(moduleLoadFromQueue+0x4a)[0x557e6567fe9a]
/opt/redis-stack/bin/redis-server *:6379(main+0x7ed)[0x557e6556fa4d]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f80267a7d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f80267a7e40]
/opt/redis-stack/bin/redis-server *:6379(_start+0x25)[0x557e6556fe05]
------ REGISTERS ------
9:M 31 Mar 2024 13:46:13.466 #
RAX:00007f80239ba7d8 RBX:00007ffd305c7370
RCX:00007f80239df230 RDX:0000000000000828
RDI:00007f80239ba7d8 RSI:0000000000000000
RBP:00007ffd305c7700 RSP:00007ffd305c7368
R8 :00007f80239ba7d8 R9 :000000000193c000
R10:0000000000000812 R11:0000000000000003
R12:0000000000000004 R13:00007ffd305c7418
R14:00007f80239bb000 R15:0000557e669a77d0
RIP:00007f8026fa51da EFL:0000000000010202
CSGSFS:002b000000000033
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7377) -> 00000000014cc000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7376) -> 00007f8000000005
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7375) -> 0000000000110000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7374) -> 0000000000001000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7373) -> 00000000014cb049
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7372) -> 00000000014cb049
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7371) -> 00000000014cc000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7370) -> 0000000000110000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736f) -> 0000000000000001
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736e) -> 0000000000000000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736d) -> 0000000000001000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736c) -> 000000000010f588
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736b) -> 000000000010f588
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c736a) -> 0000000000110000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7369) -> 0000000000000000
9:M 31 Mar 2024 13:46:13.466 # (00007ffd305c7368) -> 00007f8026f8302c
------ INFO OUTPUT ------
Server
redis_version:7.2.4
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:3a23a5ac67fe7008
redis_mode:standalone
os:Linux 5.15.133.1-microsoft-standard-WSL2 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:11.4.0
process_id:9
process_supervised:no
run_id:c971490c47bee8efc30f6ac02c854dc42827767a
tcp_port:6379
server_time_usec:1711892773465223
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:616741
executable:/opt/redis-stack/bin/redis-server
config_file:
io_threads_active:0
Clients
connected_clients:0
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
Memory
used_memory:1170024
used_memory_human:1.12M
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:1170024
used_memory_peak_human:1.12M
used_memory_peak_perc:100.81%
used_memory_overhead:184
used_memory_startup:0
used_memory_dataset:1169840
used_memory_dataset_perc:99.98%
allocator_allocated:0
allocator_active:0
allocator_resident:0
total_system_memory:4004585472
total_system_memory_human:3.73G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:-nan
allocator_frag_bytes:0
allocator_rss_ratio:-nan
allocator_rss_bytes:0
rss_overhead_ratio:-nan
rss_overhead_bytes:0
mem_fragmentation_ratio:-nan
mem_fragmentation_bytes:0
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1711892773
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
Stats
total_connections_received:0
total_commands_processed:5
instantaneous_ops_per_sec:0
total_net_input_bytes:0
total_net_output_bytes:0
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:0
total_writes_processed:0
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
eventloop_cycles:0
eventloop_duration_sum:0
eventloop_duration_cmd_sum:0
instantaneous_eventloop_cycles_per_sec:0
instantaneous_eventloop_duration_usec:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:9a3383832b0178d40c614fd48deb9de1ad5d7894
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
CPU
used_cpu_sys:0.000000
used_cpu_user:0.010919
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.010667
Modules
module:name=timeseries,ver=11011,api=1,filters=0,usedby=[],using=[],options=[]
module:name=ReJSON,ver=20609,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors]
module:name=search,ver=20812,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
module:name=RedisCompat,ver=1,api=1,filters=0,usedby=[],using=[],options=[]
module:name=bf,ver=20612,api=1,filters=0,usedby=[],using=[],options=[]
Commandstats
cmdstat_info:calls=5,usec=61,usec_per_call=12.20,rejected_calls=0,failed_calls=0
Errorstats
Latencystats
latency_percentiles_usec_info:p50=10.047,p99=31.103,p99.9=31.103
Cluster
cluster_enabled:0
Keyspace
------ CLIENT LIST OUTPUT ------
------ MODULES INFO OUTPUT ------
ReJSON_trace
ReJSON_backtrace: 0: redis_module::basic_info_command_handler
1: rejson::__info_func
2: modulesCollectInfo
at /__w/redis-stack/redis-stack/redis/src/module.c:10293:9
3: logModulesInfo
at /__w/redis-stack/redis-stack/redis/src/debug.c:1900:22
printCrashReport
at /__w/redis-stack/redis-stack/redis/src/debug.c:2183:5
4: sigsegvHandler
at /__w/redis-stack/redis-stack/redis/src/debug.c:2164:5
5:
6:
7:
8:
9:
10: _dl_catch_exception
11:
12: _dl_catch_exception
13:
14:
15: _dl_catch_exception
16: _dl_catch_error
17:
18: dlopen
19: libloading::os::unix::Library::open
20: redisgears::js_init
21: RedisModule_OnLoad
22: moduleLoad
at /__w/redis-stack/redis-stack/redis/src/module.c:12161:9
23: moduleLoadFromQueue
at /__w/redis-stack/redis-stack/redis/src/module.c:11956:13
24: main
at /__w/redis-stack/redis-stack/redis/src/server.c:7308:9
25:
26: __libc_start_main
27: _start
search_version
search_version:2.8.12
search_redis_version:7.2.4 - oss
search_index
search_number_of_indexes:0
search_fields_statistics
search_dialect_statistics
search_dialect_1:0
search_dialect_2:0
search_dialect_3:0
search_dialect_4:0
search_runtime_configurations
search_concurrent_mode:OFF
search_enableGC:ON
search_minimal_term_prefix:2
search_maximal_prefix_expansions:200
search_query_timeout_ms:500
search_timeout_policy:return
search_cursor_read_size:1000
search_cursor_max_idle_time:300000
search_max_doc_table_size:1000000
search_max_search_results:10000
search_max_aggregate_results:10000
search_search_pool_size:20
search_index_pool_size:8
search_gc_scan_size:100
search_min_phonetic_term_length:3
------ CONFIG DEBUG OUTPUT ------
slave-read-only yes
client-query-buffer-limit 1gb
io-threads-do-reads no
lazyfree-lazy-user-del no
io-threads 1
repl-diskless-load disabled
lazyfree-lazy-expire no
proto-max-bulk-len 512mb
list-compress-depth 0
lazyfree-lazy-user-flush no
sanitize-dump-payload no
replica-read-only yes
activedefrag no
repl-diskless-sync yes
lazyfree-lazy-eviction no
lazyfree-lazy-server-del no
------ FAST MEMORY TEST ------
*** Preparing to test memory region 557e6586f000 (704512 bytes)
*** Preparing to test memory region 557e66958000 (450560 bytes)
*** Preparing to test memory region 7f8020350000 (9437184 bytes)
*** Preparing to test memory region 7f80239e0000 (3670016 bytes)
*** Preparing to test memory region 7f80240f7000 (8192 bytes)
*** Preparing to test memory region 7f802465c000 (12288 bytes)
*** Preparing to test memory region 7f8024660000 (8388608 bytes)
*** Preparing to test memory region 7f8024e61000 (8388608 bytes)
*** Preparing to test memory region 7f8025c79000 (28672 bytes)
*** Preparing to test memory region 7f8025c80000 (9961472 bytes)
*** Preparing to test memory region 7f8026747000 (4096 bytes)
*** Preparing to test memory region 7f8026779000 (20480 bytes)
*** Preparing to test memory region 7f802699a000 (53248 bytes)
*** Preparing to test memory region 7f8026de8000 (12288 bytes)
*** Preparing to test memory region 7f8026f78000 (8192 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: (null) (base: (nil))
Module: /lib64/ld-linux-x86-64.so.2 (base 0x7f8026f7a000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/redis/redis/issues
If a Redis module was involved, please open in the module's repo instead.
Suspect RAM error? Use redis-server --test-memory to verify it.
Some other issues could be detected by redis-server --check-system
Bus error
The memory overhead is due to redis evicting lazily, it's intended behaviour as a performance optimization