leo-project / leofs

The LeoFS Storage System

Home Page:https://leo-project.net/leofs/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

LeoFS gateway timeout with poor network performance

francoricci opened this issue · comments

I'm running LeoFS 1.4.1 under FreeBSD.
All works wheel but from some days, I’me experimenting a timeout problem when I get files (several MB ~ 20 MB) under network slow (~ 200 KB/s).

In such situation I get a crash error from leo_gateway:

[E]     gateway02@192.168.45.100        2020-04-07 08:44:48.198266 +0200        1586241888      leo_large_object_get_handler:handle_loop/3      244     [{key,"infostud/studenti/1396786/laurea/6960c4a1-2f98-41af-87e2-e5068e193d76/6ffe8524-5eb7-41cc-a2b7-1ffe4f773990/cod_tipo_allegato_4/571345_Thesis.pdf\n2"},{index,1},{cause,timeout}]
[E]     gateway02@192.168.45.100        2020-04-07 08:44:48.199254 +0200        1586241888      null:null       0       gen_server <0.15020.6425> terminated with reason: timeout in leo_large_object_get_handler:handle_loop/3 line 247
[E]     gateway02@192.168.45.100        2020-04-07 08:44:48.199424 +0200        1586241888      null:null       0       CRASH REPORT Process <0.15020.6425> with 1 neighbours crashed with reason: timeout in leo_large_object_get_handler:handle_loop/3 line 247
[E]     gateway02@192.168.45.100        2020-04-07 08:44:48.199565 +0200        1586241888      null:null       0       Ranch listener leo_gateway_rest_api terminated with reason: timeout in leo_large_object_get_handler:handle_loop/3 line 247

I think error is in gen_server socket timeout, but where I can change this setting?
[crash.log](https://github.com/leo-project/leofs/files/4442557/crash.log)

I think to have found the solution.
In the gateway configuration file, I set:

http.sending_chunked_obj_len = 65536
large_object.reading_chunked_obj_len = 65536

It's important to set either parameters to very similar values.

Now even with slow network I can download large files. Probably with fast network is not a good performance settings but it's a balanced settings for almost all "domestic" networks.

@francoricci Yes you could reach the right solution (Congrats!).

Anyway, our document has already mentioned this topic here: https://leo-project.net/leofs/docs/faq/administration/#what-should-i-do-when-a-timeout-error-happen-during-upload-a-very-large-file

however it only mentioned the case on uploading so we should add some statements that this can apply to the downloading under a narrow bandwidth network to the document.

Thank you for reporting this!

OK. So I think issue can be closed.
Thanks.