Intermittent failures of the debug_test_boot_set test-program

[Mike Thompson]

Cirrus Logic is reporting that approximately ~40% of the 50 simulations of debug_test_boot_set test-program fail in the cv32e40p_full_covg_no_pulp regression. They fail with the following UVM_ERROR:

UVM_ERROR @ 477.300 ns : [uvmt_cv32e40p_base_test.sv](http://uvmt_cv32e40p_base_test.sv/)(345) uvm_test_top [END_OF_TEST] DUT WRAPPER virtual peripheral flagged test failure.

This indicates that the test-program has detected an error condition and signalled to the UVM environment that the test should terminate with a failure.

Type

Functionally incorrect behavior: I suspect that the random OBI stalls are interacting with the test-program in such a way that the core fails to enter debug mode when the test-program expects it to.

Steps to Reproduce

Note that this test should be run from the cv32e40p/dev branch of this repo. To run the test once:

$ git clone -b cv32e40p/dev https://github.com/openhwgroup/core-v-verif cv32e40p/dev
$ cd cv32e40p/dev/cv32e40p/sim/uvmt
$ make test SIMULATOR=<xrun|vcs|vsim> USE_ISS=NO SEED=`date +%N` TEST=debug_test_boot_set

Although you do not necessarily need to specify it as above, you must specify the SEED variable on the command-line as the default seed (1) will always pass. With a random seed you will see a failure approximately 40% of the time.

If you have an ImperasDV license you may omit USE_ISS=NO.

Additional context

I have been able to reproduce this issue with both VCS and Questasim (VSIM). It is almost certainly a test-program or verification environment issue, not an RTL issue (either way, its gotta be fixed).

[pascalgouedo]

In CV32E40Pv2 verification env, it fails even with SEED=1.
Looking at the test intent, virtual sequence and design, only way that it works every time is that fetch grant would only be set after debug_req is set.

Using the new feature we added in core-v-verif for v2 to allow to constrain OBI in command line (VSIM_USER_FLAGS=+fixed_instr_gnt_stall=10) together with SEED=1 then the test is passing every time (seems so).

[Mike Thompson]

Thanks for looking into the @pascalgouedo. We will need to add the +fixed_instr_gnt_stall=10 to the test.yaml of debug_test_boot_set. I am not 100% confident that this works for all simulators so I will investigate.