`--exit` flag does not exit with failure code

[tayls]

Expected

When the --exit flag is passed with a given threshold
And I have notifications that exceed the confidence threshold
Then the application returns a non-zero exit code indicating failure

Actual

When the --exit flag is passed with a given threshold
And I have notifications that exceed the confidence threshold
Then the application does not return a non-zero exit code indicating failure
And there is no indication that the notifications exceed the confidence threshold.

Examples

)

[tayls]

I wonder if we need to change the !is_nil(status) to !is_number(status) on line 190 of sobelow.ex

[Wai]

Same issue here with version 0.8

[Holden Oullette]

Hey @devtayls - I'm working my way through issues having just recently become maintainer, sorry for the delay. I'm trying to recreate this issue and I'm unsuccessful - meaning that I'm getting the expected behavior you outlined.

I'm just using the Sobelow codebase itself and am getting failure codes with a Low threshold (since it detects a few low confidence vulnerabilities on itself) but as soon as you bump it to Medium (where it won't find anything) it exits cleanly.

[Gonçalo Tomás]

I was also seeing this behaviour but when using mix sobelow --exit. The exit option doesn't have a boolean value but instead takes the same value of the threshold option.

Simply switching to mix sobelow --exit medium or using exit: :medium in the configuration file fixed this for me.