RDMA Tests Failing

[Nikhil Athreya]

After getting everything set up (make succeeds), doing the following results in the last 4 RDMA tests failing for me:

$ CIRRUS_SERVER_TEST_IP=<RDMA-IP> bash -c "make check"
...
Running test ./tests/object_store/test_iterator
Started server, sleeping.
Sleep finished, launching client.
Test starting
Test Started.
Starting iterator alt test.
Starting iterator array test.
Starting random prefetch test.
terminate called after throwing an instance of 'std::runtime_error'
  what():  Get took too long, likely not prefetched
Elapsed is: 14
FAIL: ./tests/test_iterator_RDMA.py
Running test ./tests/object_store/test_fullblade_store
Started server, sleeping.
Sleep finished, launching client.
Starting test.
Starting synchronous tests.
avg: 65.9
sd: 5.64712
99%: 77
Starting test sync no args.
d2.id: 42
Starting asynchronous tests.
Test async n.
Starting test of c array.
Testing nonexistent get.
Test remove starting.
test remove bulk
test shared client
test variable sizes
terminate called after throwing an instance of 'std::runtime_error'
  what():  Size not equal to the int val
FAIL: ./tests/test_store_RDMA.py
Running test ./tests/object_store/test_mt
Started server, sleeping.
Sleep finished, launching client.
terminate called after throwing an instance of 'std::runtime_error'
  what():  Incorrect value returned.
FAIL: ./tests/test_mt_RDMA.py
Running test ./tests/object_store/test_mult_clients
Started server, sleeping.
Sleep finished, launching client.
Multiple clients test
FAIL: ./tests/test_mult_clients_RDMA.py
==================================
4 of 17 tests failed
Please report to [e-mail]
==================================
Makefile:499: recipe for target 'check-TESTS' failed
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory '/data/nathreya/cirrus'
Makefile:780: recipe for target 'check-am' failed
make[1]: *** [check-am] Error 2
make[1]: Leaving directory '/data/nathreya/cirrus'
Makefile:399: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

It seems that I am able to connect to RDMA since I pass other tests like ./tests/test_client_RDMA.py but I fail these last 4 ones only. I should also note that this behavior doesn't seem to be deterministic. Last time I ran make check, only 3 of the tests failed.