Too many connections
ayushr2 opened this issue · comments
The grader keeps polling the queue on the server every two seconds or so if the server does not give it a job. Which might not be the best design. Ideally, these servers would be up all day and such requests would fill our logs with useless requests. See if one connection can be held and waited on till a job comes. Something like epoll.
This also becomes an issue while hosting the server for testing using ngrok or something because the number of connections is restricted to like 20/minute.
Ideally we want to use some sort of exponential backoff strategy as well to avoid flooding connections if we stick with the posting to an endpoing.
Another idea is to make sure that all of this communication is done over a single HTTPS connection and use the intermittent TCP heartbeating as a way of keeping track of if the connection is still alive. If TCP says its dead, it's deadm and we treat it at such -- no logic from our point, and the TCP implementation in the linux kernel is one of the most robust pieces of software out there. We'd just need to set some flags.
The added benefit is that when this does switch to HTTPS, we forgo the initial 5-way (6-way?) handshake needed to create a TLS/TCP connection.
Could you lead me to some resource which explains how we could hack into these intricacies of TCP in python? Like how we could access the heartbeat of a TCP connection?
I found this which looks promising. Does this help?
I like the idea of using just one connection to do all the communication. That would entail that we ditch all the grader endpoints. The server acts as an API only for clients then.