Resource limit: Oh No, Error running app.

[JunXiang]

Problem

1. The free model will use HuggingFaceEmbeddings which cause lots of memory.
2. The streamlit free server just have 1GB RAM.
3. When there have multiple users using free model in the same time, our app will "over the resource limits".

Reference

• Streamlit app resource limit
• Same problem in streamlit forum

Suggestion

1. improve streamlit app performance
2. RAM in streamlit
3. Select a lower embedding model of HuggingFace

[JunXiang]

• We use the windows task manager to track the memory usages.

• The quesion from stackoverflow

• Please note that even we declare model=None in app.py. While the Streamlit app's feature is that "any user interaction will cause the code to be rerun from top to bottom," Python does not immediately release resources, leading to resource consumption issues. (see stackoverflow)

• Solution

  1. Use lower embedding to reduce the memory usage.
     Commit

  2. Use st.cache_recourse to avoid memory abnormal increase.
     Commit