Intermittently seeing fleet scans get stuck
coffeegist opened this issue · comments
Frequently, when spinning up a fleet and running an axiom-scan job (could be subfinder, nuclei, etc), the job will get hung and stop receiving data back. SSHing to the instance to investigate manually shows that the job is still running, but not producing any results. This seems like a strange issue, but I've encountered it more than few times in my limited use of axiom. Has anyone else seen this? Happy to provide more data upon request, just let me know what would be helpful
Having the same issue as well. We are launching a series of axiom-scans via cron and they all seem to be hanging occasionally.
@shaniff curious, do you ever see it hang when not launching from cron? I'm launching from cron as well
@coffeegist , when I launch via shell it is successful 95% of the time. But when I do it via cron it fails the majority of the time.
Below is my cron entry...
0 6 * * * /home//shell/run.sh >> /home//axiom-cron.log 2>&1
I see the fleet coming up from the axiom-cron.log and then nothing after that:
T-Minus 1 to fleet fleet-dev-001- initialization...
Selected: [ fleet-dev-001-01 fleet-dev-001-02 fleet-dev-001-03 fleet-dev-001-04 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "fleet-dev-001-*" -f'
Interesting, but you do still see failures from normal shell. That's what I wanted to make sure of (that it's not exclusively a cron issue)
Yes, I see failures from normal shell.