Case 19.6.6.1.1 is missing severity
FLeven opened this issue · comments
invoke-hardeningKitty .\lists\finding_list_cis_microsoft_windows_11_enterprise_21h2_user.csv -EmojiSupport -Mode Audit
Output:
[*] 8/6/2022 9:25:35 AM - Starting Category Administrative Templates: System
[😺] ID 19.6.6.1.1, Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program, Result=1, Severity=Passed
Log:
"19.6.6.1.1","Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program","Passed","1"
As far as I can see this seems okay: The result of the check is 1, which is also the recommendation, therefore Passed is correct. Could you please explain me what is wrong here?
The Results gets Mixed Up, severity is Interpreted as passed, because the Case Input ist missing a severity level.
This can and will lead to false results.
A Check for complete Input, empty Input, Low,medium,high severity level could help to mitigate future errors from incomplete list entrys passed to the Test Framework.
Sorry I don't get it, there is a severity in the list: https://github.com/scipag/HardeningKitty/blob/master/lists/finding_list_cis_microsoft_windows_11_enterprise_21h2_user.csv#L6? Maybe you could give me an example of a false result to help me understand this issue?
Two results of this Test:
First one, OK -> Severity = Medium
[*] 8/6/2022 8:12:12 PM - Starting Category Administrative Templates: Start Menu and Taskbar
[$] ID 19.5.1.1, Notifications: Turn off toast notifications on the lock screen, Result=0, Recommended=1, Severity=Medium
Second Result, not OK -> Severity = passed
[*] 8/6/2022 8:12:12 PM - Starting Category Administrative Templates: System
[+] ID 19.6.6.1.1, Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program, Result=1, Severity=Passed
A severity level of passed, should be impossible
Same is found in the logfile, if you choose to create one.
But the result 1 for check 19.6.6.1.1 equals the recommendation 1, so therefore the check is passed. I decided to list all tests, positive and negative, to see what was tested in the first place
It is about consistency and if I use the exported report csv, I will have problems because of the missing fields in some rows.
I just noticed it happens on all tests, where the Name has more then one colon in it.
ckeck:
invoke-hardeningKitty .\lists\finding_list_cis_microsoft_windows_11_enterprise_21h2_machine.csv -Mode Audit
"ID","Name","Severity","Result","Recommended" -> 5 Fields😺
"19.1.3.1","Enable screen saver","Medium","","1"-> 5 Fields😺
"19.1.3.2","Password protect the screen saver","Medium","","1"-> 5 Fields😺
"19.1.3.3","Screen saver timeout","Medium","","900"-> 5 Fields😺
"19.5.1.1","Notifications: Turn off toast notifications on the lock screen","Medium","0","1"-> 5 Fields😺
"19.6.6.1.1","Internet Communication Management**:** Internet Communication Settings**:** Turn off Help Experience Improvement Program","Passed","1"-> 4 Fields😿
Ah, I see: The recommendation value is not part of the output if a test is passed. I'll change this behaviour
Old:
[😺] ID 19.6.6.1.1, Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program, Result=1, Severity=Passed
[😿] ID 19.7.4.1, Attachment Manager: Do not preserve zone information in file attachments, Result=, Recommended=2, Severity=Medium
New:
[😺] ID 19.6.6.1.1, Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program, Result=1, Recommended=1, Severity=Passed
[😿] ID 19.7.4.1, Attachment Manager: Do not preserve zone information in file attachments, Result=, Recommended=2, Severity=Medium
New (Log):
ID 19.6.6.1.1, Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program, Result=1, Recommended=1, Severity=Passed
ID 19.7.4.1, Attachment Manager: Do not preserve zone information in file attachments, Result=, Recommended=2, Severity=Medium
OK, but Severity=Passed is still wrong, it should always be LOW, MEDIUM or High ?
This is a problem if I export and import the data elsewhere.
I really with you had gone the Pester and Nunit Report way ...
It depends on the state/definition of Severity. I use severity here for the result of the test, but - if I understand you correctly - you'd like having the information what the severity of the check itself is? Would two rows help (severity result, severity check)?
If I export the data and use it elsewhere, I like to explain the data, color all failed high tests red and discuss them to be solved first etc.
If the severity level is missing from the csv, I would have to crosscheck the original lists to find out ,what severity level the test "Internet Communication Management: Internet Communication Settings: Turn off Help Experience Improvement Program" might be.
Next month we continue with all failed medium tests etc.
Gotcha! My requirement is to have a severity with {Passed,Low, Medium...}, however I can add a result_test = {Passed,Failed} and a severity_finding {Low, Medium, High, Critical} to the output, log, and report. What do you think?
Yes, please. This way the two pieces of information are unique and don't get mixed up, thanx.
I updated the format of the report file in a new commit in the development repo: 0x6d69636b/windows_hardening@9166ed8
looks good, thx.