Giters

cisagov / Malcolm

Malcolm is a powerful, easily deployable network traffic analysis tool suite for full packet capture artifacts (PCAP files), Zeek logs and Suricata alerts.

Home Page:https://cisagov.github.io/Malcolm/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

No visualization suricata log

devilman85 opened this issue · comments

commented

Suddenly I no longer have the display of suricata logs. The docker is operational but does not appear in the dashboard overview. What could this depend on?

commented

Not enough information to go on I'm afraid. Are you talking uploaded/PCAP data or live capture? Without the docker logs I don't think I could hazard a guess. I'd try restarting Malcolm and see if it resolves itself, if not and we can pin down the steps to get it to reproduce.

commented

Yes by stopping and restarting the logs came back to the dashboard. A few hours later, however, everything crashed because it said I had no space in the machine, 2TB of disk. Is it possible that having set the log rotation to 70% is not working? I reverted the virtual machine with a snapshot to restore everything

commented

Wow, 2TB of log and/or PCAP in a few hours is a lot. There was a bug that was just fixed in the latest release (#232) that dealt with calculating the amount of space indices take up, you could check the new release I put out just at the end of the week before last. The install.py --configure questions dealing with maintaining a threshold of free space are Delete the oldest indices when the database exceeds a certain size? and Should Arkime delete PCAP files based on available storage (see https://arkime.com/faq#pcap-deletion)?, the latter is also customized with the freeSpaceG value in Arkime's config.ini file (arkime/etc/config.ini in the Malcolm source tree, which, if you make changes to, you'll need to make sure is being bind-mounted into your container like this).

As your disk fills up you could try to ascertain where the storage is being used, whether in OpenSearch (du -sh ./opensearch) or PCAPs (du -sh ./pcap). Depending on the answer there, perhaps you should look into specifying some BPF filters (Capture filter (tcpdump-like filter expression; leave blank to capture all traffic) prompt in install.py --configure) to filter out storing some PCAP payloads for particular protocols/traffic you don't need to store?