SIGSEV: mbedtls::Download - I believe you're accessing a non-allocated memory chunk?
IngwiePhoenix opened this issue · comments
This should explain it all:
Ingwie@Ingwies-MBP.Speedport_W723_V_Typ_A_1_01_012 ~/W/IceTea $ lldb ./out/icetea
(lldb) target create "./out/icetea"
Current executable set to './out/icetea' (x86_64).
(lldb) process launch -- -e 'minihttp.Download("https://google.com")'
Process 47108 launched: './out/icetea' (x86_64)
-- url: https://google.com
Process 47108 stopped
* thread #1: tid = 0xc5d60d, 0x000000010000ce51 icetea`minihttp::DLSocket::_OnRequestDone(this=0x00007fff5fbfc3e0) + 33 at minihttp.cpp:1357, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x000000010000ce51 icetea`minihttp::DLSocket::_OnRequestDone(this=0x00007fff5fbfc3e0) + 33 at minihttp.cpp:1357
1354 void _OnRequestDone()
1355 {
1356 finished = true;
-> 1357 buf[bufsz] = 0; // zero-terminate
1358 }
1359
1360 void _OnRecv(void *incoming, unsigned size)
(lldb) bt
* thread #1: tid = 0xc5d60d, 0x000000010000ce51 icetea`minihttp::DLSocket::_OnRequestDone(this=0x00007fff5fbfc3e0) + 33 at minihttp.cpp:1357, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x000000010000ce51 icetea`minihttp::DLSocket::_OnRequestDone(this=0x00007fff5fbfc3e0) + 33 at minihttp.cpp:1357
frame #1: 0x0000000100004ad6 icetea`minihttp::HttpSocket::_FinishRequest(this=0x00007fff5fbfc3e0) + 86 at minihttp.cpp:1013
frame #2: 0x00000001000049e3 icetea`minihttp::HttpSocket::_OnUpdate(this=0x00007fff5fbfc3e0) + 131 at minihttp.cpp:832
frame #3: 0x0000000100003d0a icetea`minihttp::TcpSocket::update(this=0x00007fff5fbfc3e0) + 26 at minihttp.cpp:720
frame #4: 0x00000001000086ee icetea`minihttp::Download(url="https://google.com", sz=0x0000000000000000, post=0x0000000000000000) + 510 at minihttp.cpp:1393
frame #5: 0x0000000100001125 icetea`IceTeaHTTP::Download(os=0x0000000100600010, params=1, closure_values=0, need_ret_values=1, userData=0x0000000000000000) + 245 at minihttp.cpp:21
frame #6: 0x0000000100126329 icetea`ObjectScript::OS::Core::callFT(this=0x000000010200f4a8, start_pos=25, call_params=3, ret_values=1, self_for_proto=0x0000000000000000, call_enter=OS_CALLENTER_ALLOW_ONLY_ENTER, call_type=OS_CALLTYPE_AUTO, call_this_usage=OS_CALLTHIS_KEEP_STACK_VALUE) + 2249 at objectscript.cpp:26729
frame #7: 0x000000010011fb98 icetea`ObjectScript::OS::Core::execute(this=0x000000010200f4a8) + 32568 at objectscript.cpp:20618
frame #8: 0x0000000100126214 icetea`ObjectScript::OS::Core::callFT(this=0x000000010200f4a8, start_pos=21, call_params=2, ret_values=1, self_for_proto=0x0000000000000000, call_enter=OS_CALLENTER_EXECUTE_AND_RETURN, call_type=OS_CALLTYPE_FUNC, call_this_usage=OS_CALLTHIS_KEEP_STACK_VALUE) + 1972 at objectscript.cpp:26712
frame #9: 0x00000001000fb078 icetea`ObjectScript::OS::Core::callFT(this=0x000000010200f4a8, params=2, ret_values=1, call_type=OS_CALLTYPE_FUNC, call_this_usage=OS_CALLTHIS_KEEP_STACK_VALUE) + 104 at objectscript.cpp:26831
frame #10: 0x000000010013b890 icetea`ObjectScript::OS::eval(this=0x0000000100600010, str=0x00007fff5fbfec90, params=0, ret_values=1, source_code_type=OS_SOURCECODE_PLAIN, check_utf8_bom=true, handle_exception=false) + 160 at objectscript.cpp:27026
frame #11: 0x000000010013a4b6 icetea`ObjectScript::OS::eval(this=0x0000000100600010, str="minihttp.Download(\"https://google.com\")", params=0, ret_values=1, source_code_type=OS_SOURCECODE_PLAIN, check_utf8_bom=true, handle_exception=false) + 118 at objectscript.cpp:27016
frame #12: 0x00000001000a1cde icetea`IceTea::eval(this=0x0000000100600010, code=<unavailable>, expected=1) + 302 at IceTea.cpp:392
frame #13: 0x00000001000a1a1f icetea`IceTea::checkAndRunInline(this=0x0000000100600010, rt=0x00007fff5fbff6fc) + 3935 at IceTea.cpp:320
frame #14: 0x00000001000a31b1 icetea`IceTea::run(this=0x0000000100600010) + 1089 at IceTea.cpp:467
frame #15: 0x00000001000c78de icetea`main(argc=3, argv=0x00007fff5fbff9e8) + 62 at main.cpp:18
frame #16: 0x00007fff9bad65ad libdyld.dylib`start + 1
frame #17: 0x00007fff9bad65ad libdyld.dylib`start + 1
I mimiced the mbedtls API in the scripting language, and -e executes a string. From thereon out, you should see where this is going :)
Oh by the way, error only occurs on HTTPS - never on HTTP.
I will look into this fully when i have time, but won't have time until early april unfortunately...
From a quick glance google.com does a weird redirect sorta thing (sets a cookie for google.at in my case). I get the following line of trace print: "_ParseHeader: Not chunked transfer and content-length==0, this will go fail".
Explains why it crashes. But i have no idea why google is sending this weird HTTP reply.
Will push something real quick to fix the crash at least, but a better fix has to wait until later.
What. google.com sends this in the header:
alternate-protocol: 443:quic,p=1
alt-svc: quic=":443"; ma=2592000; v="31,30,29,28,27,26,25"``
Maybe there is a special header that has to be sent that quic is not supported but i can't imagine that, since HTTP is all about backwards compatibility.
Seems to be an issue specific to google, at least it didn't happen for me with any other site i tested. Might be related to a mbedtls misconfiguration on client side that google is specifically detecting, but i doubt this.
What really confuses me is that google replies with some headers but an empty body (which is why neither content-length not chunked transfer encoding is set. Makes sense but i haven't seen this sort of reply before.)
Right now i have no idea how to make the server happy to send an actual body. Try to fake a user-agent of a popular browser? Just did a cross-check with curl and that works.
I sadly didn't find out a good way to get Google to work, but found out in the process that https://api.github.com/zen also does not work - it results in the contents being NULL, triggering the "Error!" message in example 1. Any idea for that one?
Actually nevermind... that was totally an issue on my end. I ended up getting it working. :)
I had to set a User-Agent here as well. Example code:
// Example 1: Dead-simple one-shot file download API
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include "minihttp.h"
using namespace minihttp;
class DLSocket : public HttpSocket
{
public:
DLSocket() : buf(NULL), bufsz(0), bufcap(0), finished(false), fail(false){}
virtual ~DLSocket() {}
char *buf;
size_t bufsz;
size_t bufcap;
bool finished;
bool fail;
protected:
void _OnRequestDone()
{
finished = true;
buf[bufsz] = 0; // zero-terminate
}
void _OnRecv(void *incoming, unsigned size)
{
if(!size || !IsSuccess())
return;
if(bufcap + size + 1 >= bufsz) // always make sure there's 1 more byte free for the zero-terminator
{
bufcap += (bufcap / 2) + size + 1;
buf = (char*)realloc(buf, bufcap);
if(!buf)
{
fail = true;
close();
}
}
memcpy(buf + bufsz, incoming, size);
bufsz += size;
}
};
int main(int argc, char *argv[]){
char* url = "https://api.github.com/zen";
if(!InitNetwork())
return 2;
DLSocket dl;
dl.SetBufsizeIn(64 * 1024);
dl.SetNonBlocking(false);
dl.SetFollowRedirect(true);
dl.SetAlwaysHandle(false);
//dl.SetUserAgent("curl/7.40.0");
dl.Download(url, NULL, NULL, NULL);
while(dl.isOpen() || dl.HasPendingTask())
dl.update();
if(!dl.finished || dl.fail)
{
free(dl.buf);
return 2;
}
puts(dl.buf);
free(dl.buf);
return 0;
}Consider giving the minihttp::Download api an argument to set the user-agent?
No arg, just a default. That function shouldn't be used anyway ;)
Thanks for the investigation so far.
EDIT: I have no idea why (or if) the User-Agent field is required. Don't think the spec mandates it...