[Bug]: KeyError: 'included' after upgrading from 0.4.24 to 0.5.3

[Vishal Nagda]

What happened?

I'm experiencing an issue after pulling the latest changes from the main branch as suggested in issue #2440, which I raised yesterday (3rd July 2024). The application throws an exception related to the streamed response, which can be seen in the logs provided below.

Environment:

• Operating System: Ubuntu 22.04.4 LTS
• Python Version: 3.10
• chromadb Version: 0.5.3
• Other Relevant Dependencies: langchain-chroma

Steps to Reproduce:

• Follow the steps as mentioned in issue #2440.
• Pull the latest changes from the main branch.
• Run the application with the updated code.

Expected Behavior:
The application should handle the streamed response without throwing exceptions.

Actual Behavior:
The application throws a KeyError: 'included' in the chromadb/api/fastapi.py file.

Code Snippet:
Here is the relevant part of the code that leads to the exception:

# Assuming resp_json is the response from some API call
included=resp_json["included"],  # This line throws KeyError in version 0.5.3 (After pulling latest changes from main branch)

Additional Context:

• The issue started occurring after pulling the latest changes from the main branch as suggested in issue #2440.
• The application setup and environment remain unchanged from the previously working state except for the pulled changes.

Versions

• ChromaDB: 0.5.3 (Took latest pull from chromadb main branch)
• Python: 3.10
• OS: Ubuntu 22.04.4 LTS
• Other relevant dependency: langchain-chroma (0.1.2)

Relevant log output

File "/home/vishal/.local/share/virtualenvs/chat-api-ezOdHGUQ/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 195, in query
    query_results = self._client._query(
  File "/home/vishal/.local/share/virtualenvs/chat-api-ezOdHGUQ/lib/python3.10/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 146, in wrapper
    return f(*args, **kwargs)
  File "/home/vishal/.local/share/virtualenvs/chat-api-ezOdHGUQ/lib/python3.10/site-packages/chromadb/api/fastapi.py", line 522, in _query
    included=resp_json["included"],
KeyError: 'included'

[Trayan Azarov]

@vishalnagda1, thanks for reporting this. Did you upgrade both the client and server to the latest main?

[Trayan Azarov]

I just tested, and it is indeed a similar issue to #2440.

The latest client does not work with 0.5.0 or lower versions of the Chroma server.

I've made a tiny fix to make things backward-compatible #2445

[Vishal Nagda]

Hi @tazarov

Thank you for your prompt response.

To upgrade, I added the following line to my Pipfile:

chromadb = {git = "https://github.com/chroma-core/chroma.git", ref = "main"}

Then, I ran the command:

pipenv install

This should have updated ChromaDB to the latest version from the main branch. If there are additional steps required to ensure both the client and server are fully upgraded, could you please provide further guidance?

Thank you for your help.

[Trayan Azarov]

@vishalnagda1, to verify the fix, can you update your pip:

chromadb = {git = "https://github.com/chroma-core/chroma.git", ref = "trayan-07-03-fix_compatibility_with_0.5.0_and_earlier"}

[Vishal Nagda]

@tazarov I did as you suggested and it's working now.