How to use re-use properly?
denravonska opened this issue · comments
(sorry for the issue spam)
Do you have an example of how to do socket reuse properly? When I try it the first request seems to work fine:
info:1094 Program: Embeddable HTTP 1.0/1.1 client
info:1095 Version: 0.0.0
info:1096 Repo: https://github.com/howerj/httpc
info:1097 Author: Richard James Howe
info:1098 Email: howe.r.j.89@gmail.com
info:1099 Options: stk=128 tst=1 grw=1 log=1 cons=3 redirs=3 hmax=8192 sz=592
info:1103 License: The Unlicense (public domain)
info:521 domain: 212.183.159.230
info:522 port: 80
info:523 SSL: false
info:526 path /5MB.zip
debug:1312 state -- initial -> open
debug:1312 state -- open -> send-head
debug:634 custom header 'Range: bytes=0-1023' added
info:642 GET request complete
debug:1312 state -- send-head -> send-body
info:1035 no callback - nothing to do
debug:1312 state -- send-body -> recv-head
info:906 HEADER: HTTP/1.1 206 Partial Content/28
info:795 unknown field: Date: Fri, 22 Mar 2024 08:55:05 GMT
info:795 unknown field: Server: Apache
info:795 unknown field: Last-Modified: Mon, 02 Jun 2008 15:30:42 GMT
info:795 unknown field: ETag: "603fc-500000-44eb0adaf4c80"
info:739 Accept-Ranges: bytes
info:773 Content Length: 1024
info:795 unknown field: Cache-Control: no-tranform
info:795 unknown field: Content-Range: bytes 0-1023/5242880
info:795 unknown field: Keep-Alive: timeout=15, max=100
info:766 connection may be kept alive
info:795 unknown field: Content-Type: application/zip
info:922 header done
debug:1312 state -- recv-head -> recv-body
debug:1312 state -- recv-body -> done
Subsequent requests on the same client and socket seem to be thrown out of sync:
debug:1312 state -- open -> send-head
debug:634 custom header 'Range: bytes=1024-2047' added
info:642 GET request complete
debug:1312 state -- send-head -> send-body
info:1035 no callback - nothing to do
debug:1312 state -- send-body -> recv-head
info:906 HEADER: HTTP/1.1 206 Partial Content/28
info:795 unknown field: Date: Fri, 22 Mar 2024 08:55:05 GMT
info:795 unknown field: Server: Apache
info:795 unknown field: Last-Modified: Mon, 02 Jun 2008 15:30:42 GMT
info:795 unknown field: ETag: "603fc-500000-44eb0adaf4c80"
info:739 Accept-Ranges: bytes
info:773 Content Length: 1024
info:795 unknown field: Cache-Control: no-tranform
info:795 unknown field: Content-Range: bytes 1024-2047/5242880
info:795 unknown field: Keep-Alive: timeout=15, max=99
info:766 connection may be kept alive
info:795 unknown field: Content-Type: application/zip
info:922 header done
debug:1312 state -- recv-head -> recv-body
debug:1312 state -- recv-body -> done
debug:1312 state -- open -> send-head
debug:634 custom header 'Range: bytes=2048-3071' added
info:642 GET request complete
debug:1312 state -- send-head -> send-body
info:1035 no callback - nothing to do
debug:1312 state -- send-body -> recv-head
debug:218 malloc 0x55c7dc393570/256
debug:225 realloc 0x55c7dc393570/512
error:818 Got '\r' with no '\n'
error:904 protocol error (could not read first line)
debug:1312 state -- recv-head -> back-off
debug:1312 state -- back-off -> sleeps
info:663 backing off for 1000 ms, retried 1
debug:1312 state -- sleeps -> open
debug:1312 state -- open -> send-head
debug:634 custom header 'Range: bytes=2048-3071' added
info:642 GET request complete
debug:1312 state -- send-head -> send-body
info:1035 no callback - nothing to do
debug:1312 state -- send-body -> recv-head
info:906 HEADER: HTTP/1.1 206 Partial Content/28
info:795 unknown field: Date: Fri, 22 Mar 2024 08:55:06 GMT
info:795 unknown field: Server: Apache
info:795 unknown field: Last-Modified: Mon, 02 Jun 2008 15:30:42 GMT
info:795 unknown field: ETag: "603fc-500000-44eb0adaf4c80"
info:739 Accept-Ranges: bytes
info:773 Content Length: 1024
info:795 unknown field: Cache-Control: no-tranform
info:795 unknown field: Content-Range: bytes 2048-3071/5242880
info:795 unknown field: Keep-Alive: timeout=15, max=100
info:766 connection may be kept alive
info:795 unknown field: Content-Type: application/zip
info:922 header done
debug:1312 state -- recv-head -> recv-body
debug:1312 state -- recv-body -> done
debug:1312 state -- open -> send-head
debug:634 custom header 'Range: bytes=3072-4095' added
info:642 GET request complete
debug:1312 state -- send-head -> send-body
info:1035 no callback - nothing to do
debug:1312 state -- send-body -> recv-head
info:906 HEADER: ��Ļ!��x�p�i�륯�-[(9�n���^��$$:$�����pz�p����O���8��J�{n�r��O�������e���J"<�Z����-]��+)v���bռ���N�P���?�9�b#8�'�u��V��:�S��6���R��)��}�Y�ƫ�x&�bMq/182
error:856 unknown HTTP protocol/version: ��Ļ!��x�p�i�륯�-[(9�n���^��$$:$�����pz�p����O���8��J�{n�r��O�������e���J"<�Z����-]��+)v���bռ���N�P���?�9�b#8�'�u��V��:�S��6���R��)��}�Y�ƫ�x&�bMq
error:909 start line parse failed
debug:1312 state -- recv-head -> back-off
debug:1312 state -- back-off -> sleeps
info:663 backing off for 1000 ms, retried 1
^C
See provided sample file built with gcc -o http_test -g http_test.c unix.c httpc.c -lssl. I had to rename it to .txt due to github.
http_test.txt
Got it working in a test by clearing the position- and length variables of httpc_t before an operation.
diff --git a/httpc.c b/httpc.c
index bd0fcf9..c3b5506 100644
--- a/httpc.c
+++ b/httpc.c
@@ -1355,6 +1355,11 @@ static int httpc_op_heap(httpc_options_t *a, const char *url, int op, httpc_call
h->rcv_param = rcv_param;
h->snd_param = snd_param;
}
+
+ h->position = 0;
+ h->length = 0;
+ h->max = 0;
+ h->length_set = 0;
const int r = httpc_state_machine(h, url, op);
if (r != HTTPC_YIELD && r != HTTPC_REUSE)
a->state = NULL; /* make sure this is not reused */Not sure if it's the correct approach though.
Edit: I think the following will need the same re-assignment treatment, or a HEAD followed by a GET will fail since h will be reused with empty callbacks:
h->rcv = rcv;
h->snd = snd;
h->rcv_param = rcv_param;
h->snd_param = snd_param;
I'll try to have a look at this once (if) I have time over the weekend, I should probably add some more unit tests to cover these new additions as well.
I've found some more fields that would have to be moved.
diff --git a/httpc.c b/httpc.c
index c7e6b79..cb103c9 100644
--- a/httpc.c
+++ b/httpc.c
@@ -1357,12 +1357,18 @@ static int httpc_op_heap(httpc_options_t *a, const char *url, int op, httpc_call
return HTTPC_ERROR;
memset(h, 0, sizeof *h);
a->state = h;
- h->os = a;
- h->rcv = rcv;
- h->snd = snd;
- h->rcv_param = rcv_param;
- h->snd_param = snd_param;
}
+
+ h->os = a;
+ h->rcv = rcv;
+ h->snd = snd;
+ h->rcv_param = rcv_param;
+ h->snd_param = snd_param;
+ h->position = 0;
+ h->length = 0;
+ h->max = 0;
+ h->length_set = 0;
+ h->state = SM_INIT;
const int r = httpc_state_machine(h, url, op);
if (r != HTTPC_YIELD && r != HTTPC_REUSE)
a->state = NULL; /* make sure this is not reused */Two ways to reproduce when using persistent connections:
HEADfollowed byGET-> sinceHEADdoesn't have callbacks (but has allocatedh), no data callback will be used for theGETrequest- Consecutive
GETs -> length wasn't reset so httpc requested invalid data (IIRC) - Consecutive
GETs without state reset -> standard headers weren't added
Note that this is assuming httpc_op_heap is only going to be called once per request. I don't know if that's always the case (async + yield?).
Yeah, that isn't going to work when the HTTPC_YIELD option is set.