hashicorp-forge / go-test-split-action

Github Action to produce a list of go tests to run in parallel action steps using a matrix strategy

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Generating slice takes a very long time

torkelrogstad opened this issue · comments

Thanks for creating this neat action.

I'm seeing some issues where generating the slices of test names actually takes longer than running the tests themselves. On a recent run, generating the names took 3 minutes. Running the tests (half of them, through this action) took only 2 minutes.

Is there any way to debug this, and see what's going on? Alternatively, it would be nice to generate the slices once, instead of in every matrix invocation

Thanks for bringing up this issue! It sounds like a potential side effect of running go test to generate the slice that I hadn't considered until now. Ultimately, running go test can require a lot of time (downloading dependencies, go build, testmain, etc....) Could that be taking minutes of time? I don't have any other suspects within the action, but I admit I've only used this custom action on the scale of hundreds of tests.

The action log will show the exact command that is used to generate the list. By default, this is something like go test ./... -list . Would you expect that command to take minutes of time on a clean working directory? Using something like setup-go with caching can help with the build time experienced by each slice. You could experiment by substituting the go-test-split-action step for a step that just executes this command to understand how it might be sped up.

To get some more detail, you can turn on debug logging by restarting your job in the GitHub Actions UI and checking the debug logging checkbox:

Screenshot 2023-04-20 at 9 50 51 AM

You won't see timing data for each operation, but you can get a sense of which part is taking the most time.

Here is an example of the info log that shows the precise go test command that is being executed

Screenshot 2023-04-20 at 9 53 11 AM

As to your other question, generating the list in one job and using that data in another job with a matrix strategy isn't such a great experience and I think requires uploading and downloading an artifact and two separate custom actions to handle the data properly. I think that is a little beyond this scope of this action.