Giters

csirtgadgets / massive-octo-spice

DEPRECATED - USE v3 (bearded-avenger)

Home Page:https://github.com/csirtgadgets/bearded-avenger-deploymentkit/wiki

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

cif-smrt:always checking for router

liweizhe opened this issue · comments

commented

Hi,
i have use cif for several days and it worked fine before,but it stop parsing feeds yesterday.
i checked cif-smrt.log,parts of the log shows:

[2017-07-03T18:10:24,073Z][INFO]: checking for router...
[2017-07-03T18:19:20,372Z][INFO]: checking for router...
[2017-07-03T18:19:37,434Z][INFO]: checking for router...
[2017-07-03T18:34:59,643Z][INFO]: checking for router...
[2017-07-03T19:10:24,075Z][INFO]: checking for router...
[2017-07-03T19:19:20,372Z][INFO]: checking for router...

but the cif-router is already running.
in the cif-smrt log:

[2017-07-03T18:08:32,980Z][INFO]: staring up..
[2017-07-03T18:08:33,073Z][INFO]: started, waiting for messages..
[2017-07-04T08:58:43,395Z][INFO]: staring up..

I restarted the cif-services but it comes to the same situation.

commented

in addition,
the cif-smrt works fine using --testmode,
but it stop parsing feeds automatically.

commented

could be that cif-router or cif-starman is just "hung" ? did you reboot the whole box? cif-smrt tries to verify router is responding before it even tries to parse feeds, so if the http or zmq sockets aren't responding, it won't do anything...

commented

Thanks for your reply,
I am sure I have restarted both host and cif-services,but it didn't work.
The cif-starman.log shows like:

[2017-07-05T10:25:37,987Z][INFO]: starting CIF::REST
[2017-07-05T10:25:37,988Z][INFO]: starting CIF::REST
[2017-07-05T10:27:37,999Z][INFO]: starting CIF::REST
[2017-07-05T10:27:38,000Z][INFO]: starting CIF::REST
[2017-07-05T10:27:38,042Z][INFO]: generating ping request...
[2017-07-05T10:27:38,048Z][INFO]: sending ping...
[2017-07-05T10:27:38,272Z][INFO]: starting CIF::REST
[2017-07-05T10:29:38,013Z][INFO]: starting CIF::REST
[2017-07-05T10:29:38,016Z][INFO]: starting CIF::REST
[2017-07-05T10:31:38,025Z][INFO]: starting CIF::REST

Yesterday I try to use cmd "cif-smrt --testmode",after that the cif start parsing feeds automatically all day.But it stopped work at 0 am this morning.It seems there is trouble when cif try to make daily index in elasticsearch but I am not sure about that.And today it works well after I use testmode,besides,I did nothing with the cif conf before.

commented

ah, check your elasticsearch cluster health. i wonder if it's YELLOW or RED because you're making a daily index instead of monthly (which is why we moved to monthly by default, elasticsearch is fast, but it has some architectural caveats). you may need to re-scale your cluster or delete some of the older indices (if i'm understanding your setup properly..).

you can also find more ways to verify this using the "Troubleshooting CIF" FAQ:

https://github.com/csirtgadgets/massive-octo-spice/wiki/FAQ
https://github.com/csirtgadgets/massive-octo-spice/wiki/Troubleshooting-CIF

commented

Thanks for your help,
though I dont know exactly how it goes, the cif works fine after I backup es data and delete the older indices.