Down Front when use Crawler (dirb)

[Research]

Hi.

When using dirb or any other crawler, the front falls, but the container keeps working. When accessing the server, it returns 504. Serviceability is only restored when the Tanner container has restarted.

snare.err
2020-02-17 13:23:01 ERROR:aiohttp.server:log_exception: Error handling request
2020-02-17 13:28:42 ERROR:snare.tanner_handler:submit_data: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/Snare-0.3.0-py3.7.egg/snare/tanner_handler.py", line 53, in submit_data
timeout=10.0
File "/usr/lib/python3.7/site-packages/aiohttp/client.py", line 391, in _request
await resp.start(conn)
File "/usr/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 770, in start
self._continue = None
File "/usr/lib/python3.7/site-packages/aiohttp/helpers.py", line 673, in exit
raise asyncio.TimeoutError from None
concurrent.futures._base.TimeoutError
2020-02-17 13:28:42 ERROR:aiohttp.server:log_exception: Error handling request

I tried to change the timeout to 20, but it just delayed the fall of the front by 10. I also tried to bypass the helpers.py exception with "pass" did not help either. I also checked the RAM load and everything was okay. The only thing that works is a strong slowdown of crawler, but on real traffic will be much more request and it will down again. Pls Help.

Best Regards

[Evgeniia]

Hi @AndreyMironenkoGit
Please give me some more info: which tanner you are currently using? default one? your own instance?

As you can see, the problem in tanner_handler, I am pretty sure it's not a snare problem, if you are running your own tanner, please check also its logs.

[Marco Ochse]

@afeena Thanks for having a look at this. I successfully reproduced this by using dirb http://1.2.3.4. Strangely the error only appears within Snare while the Tanner instance (the one running on tcp/8090, version 0.6.0) simply crashes without errors and thus needing a manual restart.