mg_http_next_multipart bug

[herve CAUFFRIEZ]

• My goal is: uploading binary file to a mongoose server as described https://github.com/cesanta/mongoose/blob/master/examples/file-upload-html-form/main.c

• My actions were:
  struct mg_http_message *hm = (struct mg_http_message *) ev_data;
  MG_INFO(("New request to: [%.s], body size: %lu", (int) hm->uri.len,
  hm->uri.buf, (unsigned long) hm->body.len));
  printBufferHex((unsigned char)hm->body.buf, 200);
  struct mg_http_part part;
  size_t ofs = 0;
  while ((ofs = mg_http_next_multipart(hm->body, ofs, &part)) > 0) {
  MG_INFO(("Chunk name: [%.*s] filename: [%.s] length: %lu bytes",
  (int) part.name.len, part.name.buf, (int) part.filename.len,
  part.filename.buf, (unsigned long) part.body.len));
  MG_INFO(("ofs: %lu", ofs));
  printBufferHex((unsigned char)part.body.buf, 200);
  }
  mg_http_reply(c, 200, "", "Thank you!");

• My expectation was: as initially ofs = 0 at first call, part.body.buf should point to hm->body[0]

• The result I saw:
  691e 1 mongoose.c:3366:mg_iobuf_resiz 51200->53248
  691e 1 mongoose.c:1363:mg_error 5 57 OOM
  691e 2 web_server_ota.c:20:web_server New request to: [/firmware], body size: 50780
  e9 04 02 30 50 02 80 40 ee 00 00 00 0d 00 00 00 00 63 00 00 00 00 00 01 20 00 15 42 a0 ed 03 00 32 54 cd ab 00 00 00 00 00 00 00 00 00 00 00 00 30 2e 31 2e 30 2e 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 73 6d 61 6c 6c 62 72 6f 74 68 65 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 39 3a 30 31 3a 35 33 00 00 00 00 00 00 00 00 41 70 72
  20 32 34 20 32 30 32 34 00 00 00 00 00 76 35 2e 32 2e 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a6 f9 a1 cd e0 29 28 a3 f1 0e d5 05 17 98 4b ba 35 7e e2 f8 9d 99 02 50
  695a 2 web_server_ota.c:26:web_server Chunk name: [] filename: [] length: 48756 bytes
  695a 2 web_server_ota.c:29:web_server ofs: 50766
  00 00 00 00 44 65 74 65 63 74 65 64 20 69 6e 20 74 61 73 6b 20 22 00 00 22 20 61 74 20 30 78 00 53 74 61 63 6b 20 70 6f 69 6e 74 65 72 3a 20 30 78 00 00 00 53 74 61 63 6b 20 62 6f 75 6e 64 73 3a 20 30 78 00 00 00 00 20 2d 20 30 78 00 00 00 43 6f 72 65 20 00 00 00 20 72 65 67 69 73 74 65 72 20 64 75 6d 70 3a 00 55 6e 6b 6e 6f 77 6e 20 72 65 61 73 6f 6e 00 00 43 61 63 68 65 20 65 72 72 6f 72
  00 49 6e 74 65 72 72 75 70 74 20 77 64 74 20 74 69 6d 65 6f 75 74 20 6f 6e 20 43 50 55 30 00 00 00 53 74 61 63 6b 20 70 72 6f 74 65 63 74 69 6f 6e 20 66 61 75 6c 74 00 00 45 78 63 65 70 74 69 6f 6e 20 77 61

• Instead of pointing at beginning of body at first call, mg_http_next_multipart points really to file offset 7D8 ?

Environment

• mongoose version: 7.13
• Compiler/IDE and SDK: GNU 13.2.0
• Target hardware/board: ESP32C6
• Connectivity chip/module: ESP-IDF v5.2.1
• Target RTOS/OS (if applicable): ?

Thanks for your help
best regards

[Sergio R. Caprile]

• 691e 1 mongoose.c:3366:mg_iobuf_resiz 51200->53248
  691e 1 mongoose.c:1363:mg_error 5 57 OOM

"OOM" stands for Out Of Memory

What you are trying to send is larger than the memory you have available for it.
Please see our documentation, and follow the guidelines in our tutorials.
In particular: https://mongoose.ws/documentation/tutorials/file-uploads/ (this link will likely change sometime next week, we are restructuring)

[herve CAUFFRIEZ]

Thanks for the answer,
Mongoose is resizing the buffer before entering the event handler. MG_IO_SIZE is 2048, avaible memory is 252336 bytes, and 167936 bytes in one block. As the file is huge I was expecting mongoose loads only parts without trying resize buffers ?
How to prevents this resizing ?

I (816263) UTILS: Main loop running., free 252336, largest 167936
cd3d4 1 mongoose.c:3366:mg_iobuf_resiz 59392->61440
cd3d4 1 mongoose.c:1363:mg_error 9 57 OOM
I (841153) WEB_SERVER: Event Handler got HTTP_MSG, uri /firmware
cd3de 2 web_server_ota.c:20:web_server New request to: [/firmware], body size: 58972
e9 05 02 30 50 02 80 40 ee 00 00 00 0d 00 00 00 00 63 00 00 00 00 00 01 20 00 0c 42 08 e7 02 00 32 54 cd ab 00 00 00 00 00 00 00 00 00 00 00 00 30 2e 31 2e 30 2e 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 73 6d 61 6c 6c 62 72 6f 74 68 65 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 31 3a 30 32 3a 30 38 00 00 00 00 00 00 00 00 41 70 72
20 32 37 20 32 30 32 34 00 00 00 00 00 76 35 2e 32 2e 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3a da 24 3b 79 61 a0 d0 1a 49 ea c4 0d 08 4e e0 34 c6 aa 6a 9b 91 66 d0

static void ev_handler(struct mg_connection *c, int ev, void *ev_data) {
if (ev == MG_EV_HTTP_MSG) { // HTTP message

    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    ESP_LOGI(TAG, "Event Handler got HTTP_MSG, uri %.*s", (int) hm->uri.len, hm->uri.buf);

thanks

[Sergio R. Caprile]

No, Mongoose does not work the way you think; please see the docs and the tutorial.

cd3d4 1 mongoose.c:3366:mg_iobuf_resiz 59392->61440

mongoose/mongoose.c

Lines 3353 to 3367 in 57f90d2

 } else if (new_size != io->size) { 
   // NOTE(lsm): do not use realloc here. Use calloc/free only, to ease the 
   // porting to some obscure platforms like FreeRTOS 
   void *p = calloc(1, new_size); 
   if (p != NULL) { 
     size_t len = new_size < io->len ? new_size : io->len; 
     if (len > 0 && io->buf != NULL) memmove(p, io->buf, len); 
     mg_bzero(io->buf, io->size); 
     free(io->buf); 
     io->buf = (unsigned char *) p; 
     io->size = new_size; 
   } else { 
     ok = 0; 
     MG_ERROR(("%lld->%lld", (uint64_t) io->size, (uint64_t) new_size)); 
   } 

calloc() is failing.
You can increase MG_IO_SIZE to resize less often, at the cost of having unnecessarily large buffers for regular pages. https://mongoose.ws/documentation/#best-practices.
You can try with realloc().
Or, you can follow the tutorial guidance and do as other examples do.There are two choices there.
And, you can see how other examples do: https://github.com/cesanta/mongoose/tree/master/examples/file-transfer