tarantool / tarantool-qa

QA related issues of Tarantool

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

flaky test: box-luatest.gh_6794_recover_nonmatching_xlogs.test_ignore_with_force_recovery

ylobankov opened this issue · comments

From time to time I can see failures of the box-luatest.gh_6794_recover_nonmatching_xlogs.test_ignore_with_force_recovery test.

[021] box-luatest/gh_6794_recover_nonmatching_xlogs_>                 
[021] not ok 2 box-luatest.gh_6794_recover_nonmatching_xlogs.test_ignore_with_force_recovery #  
[021] Rejected result file: /tmp/tnt/rejects/box-luatest/gh_6794_recover_nonmatching_xlogs.reject
[021] [ fail ]

If we take a look at the result file that produces test-run, we will see something like this:

$ cat gh_6794_recover_nonmatching_xlogs.result
TAP version 13
1..2
# Started on Fri Feb  4 21:06:11 2022
# Starting group: box-luatest.gh_6794_recover_nonmatching_xlogs
�[35mmaster | �[31m2022-02-04 21:06:11.813 [85913] main/103/default.lua F> Can't proceed. Replicaset vclock {1: 1} doesn't match recovered data {1: 2}.�[0m
ok     1	box-luatest.gh_6794_recover_nonmatching_xlogs.test_panic_without_force_recovery
�[34mmaster | �[31m2022-02-04 21:06:12.063 [85920] main/103/default.lua F> can't initialize storage: unlink, called on fd 27, aka unix/:(socket), peer of unix/:(socket): Address already in use�[0m
not ok 2	box-luatest.gh_6794_recover_nonmatching_xlogs.test_ignore_with_force_recovery
#   .../tarantool/test/luatest_helpers/server.lua:99: Waiting for "readiness" on server master-UCg6PXluSqYt (PID 85920) timed out
#   stack traceback:
#   	.../tarantool/test/luatest_helpers/server.lua:99: in function 'wait_for_readiness'
#   	.../tarantool/test/luatest_helpers/server.lua:146: in function 'start'
#   	...t/box-luatest/gh_6794_recover_nonmatching_xlogs_test.lua:32: in function 'box-luatest.gh_6794_recover_nonmatching_xlogs.test_ignore_with_force_recovery'
#   	...
#   	[C]: in function 'xpcall'
# Ran 2 tests in 60.399 seconds, 1 success, 1 error

If we take a look at the logs of a tarantool server that is used for testing, we will see something like this:

$ cat master-UCg6PXluSqYt/master.log
2022-02-04 21:06:12.027 [85920] main/103/default.lua C> Tarantool 2.10.0-beta2-89-g2bbd13b45
2022-02-04 21:06:12.027 [85920] main/103/default.lua C> log level 5
2022-02-04 21:06:12.027 [85920] main/103/default.lua I> wal/engine cleanup is paused
2022-02-04 21:06:12.027 [85920] main/103/default.lua I> mapping 268435456 bytes for memtx tuple arena...
2022-02-04 21:06:12.028 [85920] main/103/default.lua I> Actual slab_alloc_factor calculated on the basis of desired slab_alloc_factor = 1.044274
2022-02-04 21:06:12.028 [85920] main/103/default.lua I> mapping 134217728 bytes for vinyl tuple arena...
2022-02-04 21:06:12.061 [85920] main/103/default.lua I> update replication_synchro_quorum = 1
2022-02-04 21:06:12.061 [85920] main/103/default.lua I> instance uuid 7a29efb0-4d77-48c3-b54a-1d50a3bc669c
2022-02-04 21:06:12.062 [85920] main/103/default.lua I> instance vclock {1: 1}
2022-02-04 21:06:12.063 [85920] main/103/default.lua evio.c:240 E> SocketError: unlink, called on fd 27, aka unix/:(socket), peer of unix/:(socket): Address already in use
2022-02-04 21:06:12.063 [85920] main/103/default.lua F> can't initialize storage: unlink, called on fd 27, aka unix/:(socket), peer of unix/:(socket): Address already in use