[BUG] No standard on threat object generation

[Emil]

Describe the bug

Currently there is a mix of Fields referenced for the various threat object types defined in Security content.

Creation of Shadow Copy is a good example, as it has inconsistent references:

- name: parent_process
    type: Process
    role:
    - Parent Process
  - name: process_name
    type: Process
    role:
    - Child Process

Above two observables of type=Process is listed, however the Parent Process will contain the entire command line, and the Child Process will just contain the name. As these are propagated to the generated threat_objects it can cause some weird information in the risk index.

Here is a (quite nasy) search detailing the various fields referenced for the various threat_object_types in ESCU:

| rest /services/saved/searches splunk_server=local 
| rename eai:acl.app as app title as savedsearch_name action.risk.param._risk as risk
| search savedsearch_name=ESCU*
| stats count by app savedsearch_name actions search risk
| spath input=risk path={} output=risk_object
| mvexpand risk_object
| fields - risk count
| spath input=risk_object
| stats dc(savedsearch_name) as num_searches by threat_object_field threat_object_type
| eval per_field = printf("%30s: %s searches", threat_object_field, num_searches)
| stats values(per_field) by threat_object_type

From the ESCU app in my environment:

threat_object_type=process:
Field:                         num_searches:
Image                          2 searches
ImageLoaded                    3 searches
ProcessID                      1 searches
Processes.process_name         3 searches
SourceImage                    11 searches
TargetImage                    4 searches
parent_image                   1 searches
parent_process                 2 searches
parent_process_id              2 searches
parent_process_name            146 searches
proc_terminated                1 searches
process                        5 searches
process_id                     5 searches
process_name                   196 searches

Expected behavior

I would like a bit of consistency in the information for the threat_objects within a threat_object_type.
A process threat_object should contain the same amount of information for searches that have the information.

[Bhavin Patel]

@DipsyTipsy : I am working on an effort right now to standardizing a bulk of it and will keep you posted! Do you guys use threat_object fields in your DaaC/SOC workflow ? The first part of this effort is gonna be around standarding risk_objects and then threat_objects. I would love to hear how you are leveraging threat_objects.

[Emil]

@patel-bhavin: Awesome, that is great to hear!

We're currently looking at how we can use it efficiently, and in that effort we came across the various definitions, so no concrete usecases at the moment unfortunately.

[Bhavin Patel]

Hello @DipsyTipsy: Great use case! We have made a few updates to out tooling and our yamls, namely added this check while generation of threat objects that only a certain type of fields are in the threat object types (process, url, file_name). While we would like to use CIM compliant fields names across the fleet of detections, it still remains a challenge to have an enforced standard. here's the PR for the tooling changes: splunk/contentctl#99

That said, with these changes these have been some improvements and I re ran the above query on a splunk instance with a new ESCU build and the results are better. Thanks very much for helping us build better content!

Let us know if this helps @DipsyTipsy !! thank you