[client] Add support to write connectors bundle to files and/or to queue
richard-julien opened this issue · comments
Problem to Solve
Connectors are currently doing their job like this:
- Fetch data from external sources
- Create STIX bundle from data
- Send Stix bundle to absorption queue
Its currently not easy to use this in diode environment or to get the bundle for debug.
Proposed Solution
Add options in the client python to support writing this bundles to disk and/or sending it to the queue
For this behavior we need to introduce new options.
connector:
send_to_queue: true by default
send_to_directory: false by default
send_to_directory_path: None by default (will represent the path of bundle writing)
send_to_directory_path_retention: 7 by default (number of days to keep files in the directory)
Format
Due to impersonalization, the bundle is not sufficient for later execution.
The format of the file will be a json following this shape.
{
"type": "DIRECTORY_BUNDLE",
"applicant_id": String,
"connector": { the connector information }
"entities_types": List[String],
"bundle": Bundle in JSON format,
"update": true/false,
}
How to test
Configured any connectors using the send_stix_bundle function and check the directory content.
Next step
Create a connector able to consume this directory for reabsorption
Partially covering OpenCTI-Platform/opencti#3773 for now.
Should do the job after the dedicated connector that will be able to process the directory where files are stored