Shadowserver parser may fail to autodetect report type after reload
kamil-certat opened this issue · comments
Handling feed detection
During the init
method the parser try to determine the feed type detection type [0]:
intelmq/intelmq/bots/parsers/shadowserver/parser.py
Lines 42 to 43 in 7674949
If the self.feedname
isn't set, then the mode is detect
[1]:
intelmq/intelmq/bots/parsers/shadowserver/parser.py
Lines 53 to 54 in 7674949
Later, during processing a report, the feed name is detected and eventually set [2]:
intelmq/intelmq/bots/parsers/shadowserver/parser.py
Lines 76 to 79 in 7674949
Reloading bot
Then, during the reload (not restart) happens, the bot is re-initialized without creating a new object:
Lines 271 to 285 in 7674949
This mechanism is used during log rotation by our standard Debian configuration:
intelmq/contrib/logrotate/intelmq
Line 12 in 7674949
and of course re-load the configuration, refreshing all parameters set in the config file [3]:
Lines 786 to 788 in 7674949
Failing edge case
Requirements
- Assuming you want parser to detect the feed, so you didn't set any value in
parametrs.feedname
in the parser config. - You have also installed our Debian package or manually configured log rotation with reloading.
- Your IntelMQ instance is working long enough for log rotation to occur.
Case steps
- After starting bot, the mode is correctly detect [1], and then handling every report a correct feed is detected and set [2].
- The log rotation occurs, and the parser is reloaded.
- The configuration is refreshed, overriding all fields set in the config file [3].
- Bot
init
method is called once more and tries to detect mode, butself.feedname
is still set with the last value before reload, and thefixed
mode is detected [0]. - On every next report handled, the last previously detected feed is used
Workarounds
Set the feedname
as null
in the config. However, due to the IntelMQ Manager issue with nulls certtools/intelmq-manager#294 this may be changed to an empty string and case the fail again.
Fixes
- The quickest - clean the
feedname
in bot'sshutdown
method. If the config has a value, it will be set.
However, this issue can happen to any bot relying on default values and modifying their config. Probably it should be later solved on the IntelMQ library level.
Fix added to shadowserver-dynamic-config.
Could you cherrypick it to the main develop branch? I would like to push releasing patch soon.