Server scanning gums up CDAWeb server
jbfaden opened this issue · comments
The cron job which scans the HAPI servers for their info responses and compiles the data into searchable JSON files (https://github.com/hapi-server/servers/blob/master/index/makeGiantCatalog.jy) causes problems with the CDAWeb server. I caused problems for them yesterday when running it twice while I was making changes, and without any pauses in between calls. Bernie suggested any trivial pause is not going to fix the problem.
I've disabled this scan for now.
I've checked my code for closing connections and it looks okay. I'm stress testing it against my server now.
There may be a problem on my end. Whenever there is an exception, it appears that a socket is left open:
/proc/969267/fd
spot9> ls -lrt | grep socket | wc -l
9
I was briefly getting a problem with the following Autoplot script, where it looks like there's some issue with the spaces:
#downloadResourceAsTempFile(URL('https://jfaden.net/HapiServerDemo/hapi/info?id=Iowa+City+Forecast'),1000,monitor)
print downloadResourceAsTempFile(URL('https://jfaden.net/HapiServerDemo/hapi/info?id=Iowa City Forecast'),1000,monitor)
The problem was intermittent because I assumed the 1000 meant 1000ms, when it means 1000s. If I use instead:
setScriptDescription('''This was showing strange behavior with leaving things open''')
#downloadResourceAsTempFile(URL('https://jfaden.net/HapiServerDemo/hapi/info?id=Iowa+City+Forecast'),1000,monitor)
print downloadResourceAsTempFile(URL('https://jfaden.net/HapiServerDemo/hapi/info?id=Iowa City Forecast'),-1,monitor)
it will download once every 10 seconds and shows the problem. It also appears to be leaving the socket open:
/proc/969267/fd
spot9> lsof -i | grep java
java 480609 jbf 73u IPv6 11138833 0t0 TCP ip6-localhost:39731 (LISTEN)
java 480609 jbf 357u IPv6 11191057 0t0 TCP localhost:8008->localhost:44574 (ESTABLISHED)
java 480609 jbf 378u IPv6 11210663 0t0 TCP *:8082 (LISTEN)
java 480609 jbf 469u IPv6 11187129 0t0 TCP localhost:8008 (LISTEN)
java 484230 jbf 5u IPv4 12435670 0t0 TCP *:11555 (LISTEN)
java 484230 jbf 50u IPv6 11190903 0t0 TCP *:8084 (LISTEN)
java 484230 jbf 55u IPv6 11190907 0t0 TCP *:8009 (LISTEN)
java 484230 jbf 77u IPv6 11190915 0t0 TCP localhost:8025 (LISTEN)
java 969267 jbf 52u IPv6 16987517 0t0 TCP spot9:56532->spot8:https (CLOSE_WAIT)
java 969267 jbf 54u IPv6 17015320 0t0 TCP spot9:56534->spot8:https (CLOSE_WAIT)
java 969267 jbf 58u IPv6 16985468 0t0 TCP spot9:56536->spot8:https (CLOSE_WAIT)
java 969267 jbf 59u IPv6 16985471 0t0 TCP spot9:56538->spot8:https (CLOSE_WAIT)
java 969267 jbf 60u IPv6 17015511 0t0 TCP spot9:56540->spot8:https (CLOSE_WAIT)
This has been resolved.