Warframe aggressive download causes DNS failure
will-janz opened this issue · comments
Issue Description
Works splendidly for regular Steam games, however when downloading Warframe updates, tailing access.log shows a plethora of MISSes before it randomly stops, and tailing error.log shows a ton of things like:
2019/08/19 04:34:42 [error] 1343#1343: *14553 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.0.70, server: , request: "GET /Lotus/Levels/PVP/CTFGasCityRemaster/D_pcube0.png!5B4_6zaquhnL2Bv+ruxPQgD-tQ HTTP/1.1", upstream: "http://205.185.216.42:80/Lotus/Levels/PVP/CTFGasCityRemaster/D_pcube0.png!5B4_6zaquhnL2Bv+ruxPQgD-tQ", host: "content.warframe.com"
And then a mix of that and:
2019/08/19 04:34:46 [error] 1347#1347: *14874 content.warframe.com could not be resolved (110: Operation timed out), client: 192.168.0.70, server: , request: "GET /Lotus/Levels/CorpusArenas/OutpostArena/9_pcube0.png!5B4_gkbTCZlwD0+U65sFT4Oe3A HTTP/1.1", host: "content.warframe.com"
Any DNS requests on any machine afterwards fail, but oddly but are oddly restored after I reset my router, which is pointed to the lancache machine for DNS with 1.1.1.1 as the secondary. I've configured my main machine, however, to point directly to the lancache server for DNS.
Docker Run Command
docker run --restart unless-stopped -d --name lancache-dns -p 192.168.0.55:53:53/udp -e USE_GENERIC_CACHE=true -e UPSTREAM_DNS="1.1.1.1" -e LANCACHE_IP="192.168.0.55" -e STEAMCACHE_IP="192.168.0.55 192.168.0.56 192.168.0.57 192.168.0.58 192.168.0.59" lancachenet/lancache-dns:latest
docker run --restart unless-stopped -d --name lancache -p 80:80 -v /r6/lancache/data:/data/cache -v /r6/lancache/logs:/data/logs lancachenet/monolithic:latest
docker run --restart unless-stopped -d --name sniproxy -p 443:443 lancachenet/sniproxy:latest
After resetting the router, main PC, and trying again, some interesting log entries of note:
2019/08/19 04:53:57 [error] 1344#1344: *18053 upstream prematurely closed connection while reading response header from upstream, client: 192.168.0.70, server: , request: "GET /Lotus/Levels/PVP/DMGrnOcean/1_pcube0.png!5B4_9sHw8Dy2Y5Q+rZzpjwUFjw HTTP/1.1", upstream: "http://205.185.216.42:80/Lotus/Levels/PVP/DMGrnOcean/1_pcube0.png!5B4_9sHw8Dy2Y5Q+rZzpjwUFjw", host: "content.warframe.com"
...
2019/08/19 04:56:19 [error] 1349#1349: *19107 unexpected status code 504 in slice response while sending to client, client: 192.168.0.70, server: , request: "GET /Lotus/Levels/CorpusIcePlanet/IceDefense01/8_clr.png!5A3_waCcuDpXHFwVM28PfHN+Fg HTTP/1.1", subrequest: "/Lotus/Levels/CorpusIcePlanet/IceDefense01/8_clr.png!5A3_waCcuDpXHFwVM28PfHN+Fg", upstream: "http://205.185.216.42:80/Lotus/Levels/CorpusIcePlanet/IceDefense01/8_clr.png!5A3_waCcuDpXHFwVM28PfHN+Fg", host: "content.warframe.com"
Output of docker logs lancache-dns
Executing hook /hooks/entrypoint-pre.d/10_generate_config.sh
/opt/cache-domains /scripts
From https://github.com/uklans/cache-domains
e2a0076..5ab391a master -> origin/master
HEAD is now at 5ab391a Domain correction for Riot's new chunking patcher (#105)
/scripts
_ _ _
| | | | | |
___| |_ ___ __ _ _ __ ___ ___ __ _ ___| |__ ___ _ __ ___| |_
/ __| __/ _ \/ _` | '_ ` _ \ / __/ _` |/ __| '_ \ / _ \ | '_ \ / _ \ __|
\__ \ || __/ (_| | | | | | | (_| (_| | (__| | | | __/_| | | | __/ |_
|___/\__\___|\__,_|_| |_| |_|\___\__,_|\___|_| |_|\___(_)_| |_|\___|\__|
configuring /etc/resolv.conf to stop from looping to ourself
Bootstrapping DNS from https://github.com/uklans/cache-domains.git
----------------------------------------------------------------------
Using Generic Server: 192.168.0.55
Make sure you are using a monolithic cache or load balancer at 192.168.0.55
----------------------------------------------------------------------
Processing service: apple
Enabling service with ip(s): 192.168.0.55
Processing service: arenanet
Enabling service with ip(s): 192.168.0.55
Processing service: blizzard
Enabling service with ip(s): 192.168.0.55
Processing service: daybreak
Enabling service with ip(s): 192.168.0.55
Processing service: epicgames
Enabling service with ip(s): 192.168.0.55
Processing service: frontier
Enabling service with ip(s): 192.168.0.55
Processing service: hirez
Enabling service with ip(s): 192.168.0.55
Processing service: nexusmods
Enabling service with ip(s): 192.168.0.55
Processing service: neverwinter
Enabling service with ip(s): 192.168.0.55
Processing service: nintendo
Enabling service with ip(s): 192.168.0.55
Processing service: renegadex
Enabling service with ip(s): 192.168.0.55
Processing service: riot
Enabling service with ip(s): 192.168.0.55
Processing service: rockstar
Enabling service with ip(s): 192.168.0.55
Processing service: sony
Enabling service with ip(s): 192.168.0.55
Processing service: steam
Enabling service with ip(s): 192.168.0.55 192.168.0.56 192.168.0.57 192.168.0.58 192.168.0.59
Processing service: uplay
Enabling service with ip(s): 192.168.0.55
Processing service: teso
Enabling service with ip(s): 192.168.0.55
Processing service: twitch
Enabling service with ip(s): 192.168.0.55
Processing service: warframe
Enabling service with ip(s): 192.168.0.55
Processing service: wargaming
Enabling service with ip(s): 192.168.0.55
Processing service: wsus
Enabling service with ip(s): 192.168.0.55
Processing service: xboxlive
Enabling service with ip(s): 192.168.0.55
---
finished bootstrapping.
Executing hook /hooks/supervisord-pre.d/10_config_check.sh
checking Bind9 config
2019-08-19 04:08:33,720 CRIT Set uid to user 0
(Side note, my comment failed to send after reproducing this).
Temporary work-around: in the Warframe client, disabling aggressive download seems to work perfectly. Tailing access.log showed a much slower stream of requests (as expected), which leads me to believe something somewhere is getting overwhelmed.
Can you confirm that the aggressive download mode works on your network (with your router) without lancache?
Tried several times with fresh Warframe installs using the lancache to speed up the initial download through Steam.
(NIC's: 1.1.1.1, router: lancache) No issues.
(NIC: auto DNS, router: lancache), and with it primed, aggressive mode looks like it worked, all cache hits, but DNS still failed with "Temporary failure in name resolution" when trying to ping any website.
(NIC: auto DNS, router: 1.1.1.1) Warframe with aggressive downloads updates without issue.
Thanks for the issue report. I suspect that this may take quite some investigating to track down.
I'm also concerned that it might be your router that's interfering, although your tests make me doubt that I can't be sure. Is the issue still present if you set your NIC to use lancache, and set lancahe to use 1.1.1.1 or 8.8.8.8 as it's upstream dns?
Lancache has the upstream DNS is set to 1.1.1.1 (-e UPSTREAM_DNS="1.1.1.1", right?) I'm not ruling my router out either, but all other services work flawlessly.
We default to 8.8.8.8 but yes -e UPSTREAM_DNS="1.1.1.1" is the correct syntax.
I reuploaded my router's firmware and did a factory reset and reconfigured everything but DOS protections, tested, no issue, but resolving DNS took about 5 seconds as opposed to maybe 100ms but only on some machines. Configured DOS protections, redownloaded Warframe again, and it still worked but with these errors:
2019/08/27 01:45:19 [error] 1345#1345: *35861 upstream prematurely closed connection while reading response header from upstream, client: 192.168.0.103, server: , request: "GET /depot/496106/manifest/5638279466685264270/5 HTTP/1.1", upstream: "http://162.254.195.43:80/depot/496106/manifest/5638279466685264270/5", host: "cache18-lax1.steamcontent.com"
2019/08/27 01:45:24 [error] 1349#1349: *35853 upstream prematurely closed connection while reading response header from upstream, client: 192.168.0.103, server: , request: "GET /depot/223851/chunk/116e23c4100260676b01009060539bc344885e03 HTTP/1.1", upstream: "http://162.254.193.50:80/depot/223851/chunk/116e23c4100260676b01009060539bc344885e03", host: "cache16-ord1.steamcontent.com"
DNS still resolved fine and no router reboot was required. I'm gonna chalk this up to bugged out router firmware and close for now. Thank you for your time!