Needed in the IDF (intelmq data format): severity

Question

Needed in the IDF (intelmq data format): severity

aaronkaplan opened this issue a year ago · comments

need a "severity" field. Shadowserver is going to add that soon.
Should have documentation on what this means for the end-user / recipient (for example: severity:critical => RCE)
Have a guide on how to use the values and how to act on it for shadowserver feeds in particular

Cross check with @elsif2

Sebastian · Answer 1 · Wed May 24 2023 19:11:18 GMT+0800 (China Standard Time)

The criticality of an event depends not only on the type of event defined by the feed provider but also on criteria known to the IntelMQ operator, such as which network the device is in.

How much will the criticality be used? Is it worth a new field (involves adapting all outputs on all instances), or is it for extra?

Sebastian · Answer 2 · Wed May 24 2023 19:12:37 GMT+0800 (China Standard Time)

P.S.: We normally follow semantic versioning, so 3.1.1 (the milestone you selected for this issue) is a patch/bugfix release. IMHO a newly added field is not a patch.

AaronK · Answer 3 · Wed May 24 2023 20:42:42 GMT+0800 (China Standard Time)

It is embedded in a bigger discussion with Shadowserver and should reflect the criticality of the vulnerability as estimated by the source (shadowserver in this case). So maybe we need to name the proposed field better (such as "source.crticicality" or so?) but that's the idea. Of course, the recipient will have to define his own criticality but it does convey something if the source says it's critical (for example a RCE). Shadowserver will add that to all their feeds if I understood them correctly. And I think that's actually generic enough for all our feeds to have that field here.

…

On 24.05.2023, at 13:11, Sebastian ***@***.***> wrote: The criticality of an event depends not only on the type of event defined by the feed provider but also on criteria known to the IntelMQ operator, such as which network the device is in. How much will the criticality be used? Is it worth a new field (involves adapting all outputs on all instances), or is it for extra? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were assigned.

Sebastian · Answer 4 · Wed May 24 2023 21:09:56 GMT+0800 (China Standard Time)

I think we should get some feedback from users (mailinglist?) as this change has an impact on their setups.

elsif2 · Answer 5 · Wed May 24 2023 21:46:52 GMT+0800 (China Standard Time)

Shadowserver plans to add a severity field to every event after the https://github.com/certtools/intelmq/tree/shadowserver-dynamic-config branch is integrated into the next release. I plan to submit the PR next week.

Severity levels:

critical (typically for highly critical vulnerabilities that are being actively exploited, where failure to remediate poses a very high likelyhood of compromise. typically for RCE or changing or extracting sensitive data)
high (end of life systems, internal systems that you can log into with authentication that are meant to be internal (SMB, RDP), some data can be leaked. Drones/sinkhole events end up in this category)
medium (risk of participating in DDoS, unencrypted services requiring login, vulnerabilities requiring MITM to exploit, attacker will need to know internal systems/infrastructure in order to exploit it)
low (deviation from best practice - little to no practical way to exploit, but setup is not ideal)
info (Informational only, no concerns whatsoever)

AaronK · Answer 6 · Wed May 24 2023 21:55:28 GMT+0800 (China Standard Time)

I think it does not impact their setups to have an additional field in the IDF format: if their bots don't know the field, then nothing will happen. We however should add a tutorial on how to use that new field for the shadowserver feeds. Because - there it's going to come - for a good reason actually. Based on user feedback (from what I gained from talking to Piotr). Best, a.

…

On 24.05.2023, at 15:10, Sebastian ***@***.***> wrote: I think we should get some feedback from users (mailinglist?) as this change has an impact on their setups. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were assigned.

Sebastian · Answer 7 · Wed May 24 2023 21:57:19 GMT+0800 (China Standard Time)

I think it does not impact their setups to have an additional field in the IDF format

When an output bot writes data to a database, it does. I think writing data to a database is not uncommon.

AaronK · Answer 8 · Wed May 24 2023 21:59:50 GMT+0800 (China Standard Time)

Yes, but having a bot write all fields (without specifically selecting which fields it needs ) to some output DB sounds like a bug at that part, no? Anyways, I got your point. We'll make an IEP008 for that...

Sebastian · Answer 9 · Wed May 24 2023 22:07:48 GMT+0800 (China Standard Time)

Yes, but having a bot write all fields (without specifically selecting which fields it needs ) to some output DB sounds like a bug at that part, no?

All output bots except the SMTP Output bot write all fields to their destination. Complete list is at https://intelmq.readthedocs.io/en/latest/user/bots.html#output-bots

There are some expert bots to remove fields though. The simplest is the Field Reducer Bot.

AaronK · Answer 10 · Wed May 24 2023 22:10:45 GMT+0800 (China Standard Time)

On 24.05.2023, at 16:07, Sebastian ***@***.***> wrote: Yes, but having a bot write all fields (without specifically selecting which fields it needs ) to some output DB sounds like a bug at that part, no? All output bots except the SMTP Output bot write all fields to their destination. Complete list is at https://intelmq.readthedocs.io/en/latest/user/bots.html#output-bots There are some expert bots to remove fields though. The simplest is the Field Reducer Bot.

I think we have a problem then with output bots ;-) It's the equivalent of doing a SELECT * FROM ... and then wondering that you get more fields than you expected hehe.