vegaprotocol / jenkins-shared-library

Jenkins shared library for Vega pipeline needs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Create job to run network-level tests

jgsbennett opened this issue · comments

We need a new Jenkins job that will run system tests which take control of the network. Currently, I'm imagining we can use the mark "network_infra" (to avoid clashing with net params or similar. Actual mark still very much up for debate) to select appropriate tests.

Justification:
Currently, all of the existing "feature" tests could be run equally well on 3 nodes, 15 nodes, and for the most part, for varying network parameter settings, and we may well choose to do this in future. It seems sensible to separate the tests that have very specific requirements for network-level entities, which they wish to handle.

I've been running them locally as follows:

# Set up network as per normal
# Currently also start network, but we could yet make the tests responsible for that initial start
pytest -m network_infra

^ but I expect we'll actually want to run them in the same "make run-tests" format as the other tests.

Additional requirements:

  • Logs from all nodes (I believe Daniel indicated these should all be picked up automatically? Including before/after network starts/stops, checkpoints, node additions, etc ?
  • We should probably additional export artifacts from some extra directory, so that the tests can export other relevant data (potentially snapshot/checkpoint data, for instance). The existing tests use the "test_logs" dir, so we should probably reuse this.

Additional note:
At time of writing, it is necessary to run these tests with v1 of data node, or they fail. (Bug to be raised against data node for this).