nginx gives 400 bad request when using psiturk with jspsych
mvdoc opened this issue · comments
@adamliter I'm getting a Bad Request error 400 when using psiturk with jspsych. It works fine if I bypass nginx and access gunicorn directly, so there's something happening with nginx. I tried looking up on google and changed a bunch of parameters (mostly related to increasing the buffer size), but it still fails.
I have created a minimal example that fails with nginx, which you can download here: https://www.dropbox.com/s/kk5q8vo0qh6wzq2/nginx_fail.zip
Enabling debugging for nginx, this is the error I get:
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy status 400 "400 BAD REQUEST"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Server: gunicorn/19.4.5"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Date: Tue, 19 Sep 2017 21:54:34 GMT"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Connection: close"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Content-Type: text/html"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Content-Length: 192"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header done
2017/09/19 21:54:34 [debug] 6#6: *1 HTTP/1.1 400 BAD REQUEST
Server: nginx/1.13.5
Date: Tue, 19 Sep 2017 21:54:34 GMT
Content-Type: text/html
Content-Length: 192
Connection: keep-alive
And this is the complete log
2017/09/19 21:54:34 [debug] 6#6: *1 http process request line
2017/09/19 21:54:34 [debug] 6#6: *1 http request line: "GET /complete?uniqueId=debugRD825H:debug02HM1Q HTTP/1.1"
2017/09/19 21:54:34 [debug] 6#6: *1 http uri: "/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 http args: "uniqueId=debugRD825H:debug02HM1Q"
2017/09/19 21:54:34 [debug] 6#6: *1 http exten: ""
2017/09/19 21:54:34 [debug] 6#6: *1 posix_memalign: 000000A63D7362B0:4096 @16
2017/09/19 21:54:34 [debug] 6#6: *1 http process request header line
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Host: 127.0.0.1"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Connection: keep-alive"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Upgrade-Insecure-Requests: 1"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.91 Safari/537.36"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Referer: http://127.0.0.1/exp?hitId=debugOD8KFK&assignmentId=debug02HM1Q&workerId=debugRD825H&mode=debug"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Accept-Encoding: gzip, deflate, br"
2017/09/19 21:54:34 [debug] 6#6: *1 http header: "Accept-Language: en-US,it-IT;q=0.8,it;q=0.6,en;q=0.4"
2017/09/19 21:54:34 [debug] 6#6: *1 http header done
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 0
2017/09/19 21:54:34 [debug] 6#6: *1 rewrite phase: 1
2017/09/19 21:54:34 [debug] 6#6: *1 test location: "/"
2017/09/19 21:54:34 [debug] 6#6: *1 using configuration "/"
2017/09/19 21:54:34 [debug] 6#6: *1 http cl:-1 max:1048576
2017/09/19 21:54:34 [debug] 6#6: *1 rewrite phase: 3
2017/09/19 21:54:34 [debug] 6#6: *1 post rewrite phase: 4
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 5
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 6
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 7
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 8
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 9
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 10
2017/09/19 21:54:34 [debug] 6#6: *1 post access phase: 11
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 12
2017/09/19 21:54:34 [debug] 6#6: *1 try files handler
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 trying to use file: "/complete" "/var/www/exp/nginx_fail/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 trying to use dir: "/complete" "/var/www/exp/nginx_fail/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 trying to use file: "@proxy_to_psiturk" "/var/www/exp/nginx_fail@proxy_to_psiturk"
2017/09/19 21:54:34 [debug] 6#6: *1 test location: "@proxy_to_psiturk"
2017/09/19 21:54:34 [debug] 6#6: *1 using location: @proxy_to_psiturk "/complete?uniqueId=debugRD825H:debug02HM1Q"
2017/09/19 21:54:34 [debug] 6#6: *1 rewrite phase: 3
2017/09/19 21:54:34 [debug] 6#6: *1 post rewrite phase: 4
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 5
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 6
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 7
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 8
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 9
2017/09/19 21:54:34 [debug] 6#6: *1 access phase: 10
2017/09/19 21:54:34 [debug] 6#6: *1 post access phase: 11
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 12
2017/09/19 21:54:34 [debug] 6#6: *1 generic phase: 13
2017/09/19 21:54:34 [debug] 6#6: *1 http init upstream, client timer: 0
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "Host"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "127.0.0.1"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "X-Forwarded-Host"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "localhost"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "X-Real-IP"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "172.18.0.1"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "X-Forwarded-For"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "172.18.0.1"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "Referer"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "http://127.0.0.1/exp?hitId=debugOD8KFK&assignmentId=debug02HM1Q&workerId=debugRD825H&mode=debug"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "X-Forwarded-Proto"
2017/09/19 21:54:34 [debug] 6#6: *1 http script var: "http"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "Connection"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: "close"
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: ""
2017/09/19 21:54:34 [debug] 6#6: *1 http script copy: ""
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Upgrade-Insecure-Requests: 1"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.91 Safari/537.36"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Accept-Encoding: gzip, deflate, br"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Accept-Language: en-US,it-IT;q=0.8,it;q=0.6,en;q=0.4"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header:
"GET /complete?uniqueId=debugRD825H:debug02HM1Q HTTP/1.0
Host: 127.0.0.1
X-Forwarded-Host: localhost
X-Real-IP: 172.18.0.1
X-Forwarded-For: 172.18.0.1
Referer: http://127.0.0.1/exp?hitId=debugOD8KFK&assignmentId=debug02HM1Q&workerId=debugRD825H&mode=debug
X-Forwarded-Proto: http
Connection: close
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.91 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,it-IT;q=0.8,it;q=0.6,en;q=0.4
"
2017/09/19 21:54:34 [debug] 6#6: *1 http cleanup add: 000000A63D736E10
2017/09/19 21:54:34 [debug] 6#6: *1 get rr peer, try: 1
2017/09/19 21:54:34 [debug] 6#6: *1 stream socket 12
2017/09/19 21:54:34 [debug] 6#6: *1 epoll add connection: fd:12 ev:80002005
2017/09/19 21:54:34 [debug] 6#6: *1 connect to 172.18.0.4:22362, fd:12 #8
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream connect: -2
2017/09/19 21:54:34 [debug] 6#6: *1 posix_memalign: 000000A63D74E220:128 @16
2017/09/19 21:54:34 [debug] 6#6: *1 event timer add: 12: 60000:1505858134101
2017/09/19 21:54:34 [debug] 6#6: *1 http finalize request: -4, "/complete?uniqueId=debugRD825H:debug02HM1Q" a:1, c:3
2017/09/19 21:54:34 [debug] 6#6: *1 http request count:3 blk:0
2017/09/19 21:54:34 [debug] 6#6: *1 http finalize request: -4, "/complete?uniqueId=debugRD825H:debug02HM1Q" a:1, c:2
2017/09/19 21:54:34 [debug] 6#6: *1 http request count:2 blk:0
2017/09/19 21:54:34 [debug] 6#6: *1 http run request: "/complete?uniqueId=debugRD825H:debug02HM1Q"
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream check client, write event:1, "/complete"
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream request: "/complete?uniqueId=debugRD825H:debug02HM1Q"
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream send request handler
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream send request
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream send request body
2017/09/19 21:54:34 [debug] 6#6: *1 chain writer buf fl:1 s:656
2017/09/19 21:54:34 [debug] 6#6: *1 chain writer in: 000000A63D736E50
2017/09/19 21:54:34 [debug] 6#6: *1 writev: 656 of 656
2017/09/19 21:54:34 [debug] 6#6: *1 chain writer out: 0000000000000000
2017/09/19 21:54:34 [debug] 6#6: *1 event timer del: 12: 1505858134101
2017/09/19 21:54:34 [debug] 6#6: *1 event timer add: 12: 60000:1505858134104
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream request: "/complete?uniqueId=debugRD825H:debug02HM1Q"
2017/09/19 21:54:34 [debug] 6#6: *1 http upstream process header
2017/09/19 21:54:34 [debug] 6#6: *1 malloc: 000000A63D7372C0:4096
2017/09/19 21:54:34 [debug] 6#6: *1 recv: eof:0, avail:1
2017/09/19 21:54:34 [debug] 6#6: *1 recv: fd:12 155 of 4096
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy status 400 "400 BAD REQUEST"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Server: gunicorn/19.4.5"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Date: Tue, 19 Sep 2017 21:54:34 GMT"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Connection: close"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Content-Type: text/html"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header: "Content-Length: 192"
2017/09/19 21:54:34 [debug] 6#6: *1 http proxy header done
2017/09/19 21:54:34 [debug] 6#6: *1 HTTP/1.1 400 BAD REQUEST
Server: nginx/1.13.5
Date: Tue, 19 Sep 2017 21:54:34 GMT
Content-Type: text/html
Content-Length: 192
Connection: keep-alive
Do you have any clue on how to fix this?
Ping @adamliter and perhaps @deargle?
Hi @mvdoc. Yes, sorry, this is in on my radar. I'm hoping to get a chance to look at it this weekend. I've been a bit swamped this week. Sorry!
Thanks @adamliter, I managed to debug it and solved the problem. Updating psiturk.js with the latest version did it. The reason was that at the end of the experiment with the old psiturk.js, psiturk would request
self.completeHIT = function() {
self.teardownTask();
// save data one last time here?
window.location= self.taskdata.adServerLoc + "?uniqueId=" + self.taskdata.id;
}
instead of (new version)
self.completeHIT = function() {
self.teardownTask();
// save data one last time here?
window.location= self.taskdata.adServerLoc + "?uniqueId=" + self.taskdata.id + "&mode=" + self.taskdata.mode;
}
so because of the missing &mode=, nginx wouldn't serve the page (I don't know why though, would be curious to know why).
I'm gonna close this. Thanks again!
It looks like that change was committed by @deargle, so he can probably answer better. But it looks like this is an indicator of whether you're in live or debug mode. Maybe without this indicator, it was trying to save data to the live version of things while you were in debug mode?
It worked when you used gunicorn because psiturk.js is loaded dynamically, from the up-to-date pip repo. But when you use nginx it serves it statically without asking flask/gunicorn to load it, because you've copied it into a static folder (why did you copy it into a static dir?). Flask/gunicorn explodes when you don't pass mode because the route tries to access it
Now it all makes sense. Thank you so much @deargle @adamliter!
why did you copy it into a static dir?
I have no idea ¯_(ツ)_/¯ probably a residual of when I was playing around...