Pagination and Rate Limit Issues in File Viewer When Handling Larger VectorStores

[Bailey Simrell]

Issue Overview

I am experiencing an issue where only a subset (20 out of 300) of the files in my vectorstore is visible in the UI of the file viewer component. Despite attempts to resolve this by fetching additional data and handling pagination, I keep encountering rate limit errors that hinder further data fetching.

Steps to Reproduce

1. Populate the vectorstore with more than 20 files.
2. Access the file viewer UI which fetches and displays the files.
3. Notice that only the first 20 files are displayed, and subsequent fetch attempts either result in repeated data (same 20 files) or hit rate limits.

Expected Behavior

The file viewer should correctly paginate through all files in the vectorstore, displaying all (n) files without hitting rate limits prematurely.

Actual Behavior

Only 20 files are displayed repeatedly, and attempts to fetch more files frequently hit the rate limit, even with extremely conservative fetch intervals (e.g., every 30 seconds).

Possible Solutions or Suggestions

• Pagination Handling: It seems that the API might be missing proper pagination handling to load subsequent files beyond the initial batch.
• Rate Limit Management: There might be an issue with how rate limits are managed, or possibly the limits are too stringent for practical use in this scenario. Adjusting the rate limit policy or providing guidelines on managing fetches could be beneficial.

Additional Context

Here's the logic I've tried implementing to handle fetching, with adjustments for rate limits:

const fetchFiles = async (retryDelay = 1000) => {
  try {
    const resp = await fetch("/api/assistants/files", { method: "GET" });
    if (resp.status === 429) {  
      setTimeout(() => fetchFiles(retryDelay * 2), retryDelay);  // Exponential backoff
      return;
    }
    const data = await resp.json();
    setFiles(data);
  } catch (error) {
    console.error('Failed to fetch files:', error);
  }
};

Any insights or suggestions on how to properly paginate and handle rate limits in this scenario would be greatly appreciated!

[Divyansh Verma]

I have made changes to the app you can fork it here and check it https://github.com/Ghastly2001/openai-assistants-quickstart

[Ilan Bigio]

@Ghastly2001 PRs welcome!

[LDD19]

This should be higher up. I left a few tabs open and burned through over 70gb of bandwidth in only 2 days.