[Feature Request] support limit model file load speed
hiberabyss opened this issue · comments
In my case, the model directory is mounted from network like nfs.
When load new model file, the network capacity was used too much which may cause some requests failed.
Is it possible to limit the read speed of new model file?
@hiberabyss, Thank you for raising this feature request.
@alekwang, Please take a look into this feature request. Is there an option available to limit network bandwidth to load models in TF serving. Thank you!
We discussed on this feature internally and we couldn't find any option to implement this at this point of time.
As I went through other options for your use case, I came across --limit-rate param in wget to limit the file retrieval rate. Also, there are multiple packages on linux to limit network bandwidth. Ref: limit network bandwidth on linux.
Ref: Limit File Download Speed Using Wget.
Hope this helps. Thank you!
We discussed on this feature internally and we couldn't find any option to implement this at this point of time. As I went through other options for your use case, I came across --limit-rate param in wget to limit the file retrieval rate. Also, there are multiple packages on linux to limit network bandwidth. Ref: limit network bandwidth on linux. Ref: Limit File Download Speed Using Wget.
Hope this helps. Thank you!
Do tensorflow serving use wget to fetch model files?
If you are copying files from other sources (github or gcp bucket) to linux machine where tf serving is installed, then wget can be used in that case. One suggestion is to use cloud bucket to load your model to mitigate network capacity issue.
Thank you!
If you are copying files from other sources (github or gcp bucket) to linux machine where tf serving is installed, then wget can be used in that case. One suggestion is to use cloud bucket to load your model to mitigate network capacity issue.
Thank you!
Ali ossfs mount was used in my case.
@hiberabyss, Thank you for the feature request. We discussed on this feature internally and we couldn't find any option to implement the functionality at this point of time.
Please refer limit network bandwidth on linux if it helps. Thank you!
@hiberabyss, Thank you for the feature request. We discussed on this feature internally and we couldn't find any option to implement the functionality at this point of time.
Please refer limit network bandwidth on linux if it helps. Thank you!
What if I send PR to add new option for limiting read speed?
@hiberabyss, We are happy to review PR and merge changes bought by the community. Please let us know once you submit the PR and we will work on it. Thank you!
Closing this due to inactivity. Please take a look into the answers provided above, feel free to reopen and post your comments(if you still have queries on this). Thank you!