Crash after upgrading to 2.8.0 from 2.7.0
SoniCoder opened this issue · comments
Encountering the following crash post upgrade from 2.7.0 to 2.8.0 and had to revert the upgrade:
2021-03-08 10:31:35.812 [error] <0.13212.0>@cowboy_req:reply:812 CRASH REPORT Process <0.13212.0> with 0 neighbours crashed with reason: no match of right hand value 2 in cowboy_req:reply/4 line 812 2021-03-08 10:31:35.813 [error] <0.13212.0>@cowboy_req:reply:812 Cowboy stream 1 with ranch listener butler_base_http and connection process <0.13211.0> had its request process exit with reason: no match of right hand value 2 in cowboy_req:reply/4 line 812
2021-03-08 10:31:35 =CRASH REPORT====
crasher:
initial call: cowboy_stream_h:request_process/3
pid: <0.13212.0>
registered_name: []
exception error: {{badmatch,2},[{cowboy_req,reply,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_req.erl"},{line,812}]},{butler_pps_api_handler,api_seat_login,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/apps/pf/src/interfaces/in/butler_pps_api_handler.erl"},{line,508}]},{cowboy_rest,call,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,1575}]},{cowboy_rest,process_content_type,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,1096}]},{cowboy_rest,upgrade,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,284}]},{cowboy_stream_h,execute,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_stream_h.erl"},{line,300}]},{cowboy_stream_h,request_process,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_stream_h.erl"},{line,291}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
ancestors: [<0.13211.0>,<0.3629.0>,<0.3628.0>,ranch_sup,<0.2229.0>]
message_queue_len: 0
messages: []
links: [<0.13211.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 6772
stack_size: 28
reductions: 34973
neighbours:
2021-03-08 10:31:35 =ERROR REPORT====
Ranch listener butler_base_http, connection process <0.13211.0>, stream 1 had its request process <0.13212.0> exit with reason {badmatch,2} and stacktrace [{cowboy_req,reply,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_req.erl"},{line,812}]},{butler_pps_api_handler,api_seat_login,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/apps/pf/src/interfaces/in/butler_pps_api_handler.erl"},{line,508}]},{cowboy_rest,call,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,1575}]},{cowboy_rest,process_content_type,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,1096}]},{cowboy_rest,upgrade,4,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_rest.erl"},{line,284}]},{cowboy_stream_h,execute,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_stream_h.erl"},{line,300}]},{cowboy_stream_h,request_process,3,[{file,"/var/lib/jenkins/workspace/ButlerPipeline_ButlerServer-1.5-Build-commit-id/_build/default/lib/cowboy/src/cowboy_stream_h.erl"},{line,291}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]
This commit is probably responsible: 39b2816
Yes that's the commit. But I'm not sure what you want me to fix, your code is incorrect, the only problem in Cowboy is the unhelpful error message.
This maybe present at a lot of places in our tech stack so fixing everywhere may not be viable and this will prevent us from upgrading cowboy. I wish if there was a much nicer backwards compatible way to do this
You can make your own cowboy_req:reply
function if you really want this behavior. But sending bodies with these status codes is not correct so it may break again later on.
The error message has been improved and will be in 2.11. Closing, thanks!