Tensor not found in Tensorflow module
erlebach opened this issue · comments
Initial Checks
- I have read and followed the docs and still think this is a bug
Description
I am using Poetry, and I added the docarray
module, version 0.39 . Here is the pyproject.toml
I am working with:
[tool.poetry]
name = "langchain-monorepo"
version = "0.0.1"
description = "LangChain mono-repo"
authors = []
license = "MIT"
readme = "README.md"
repository = "https://www.github.com/langchain-ai/langchain"
[tool.poetry.dependencies]
python = ">=3.9,<4.0"
python-dotenv = "^1.0.0"
openai = "^0.28.0"
chromadb = "^0.4.12"
tiktoken = "^0.5.1"
huggingface-hub = "^0.17.2"
langchain = "^0.0.300"
redlines = "^0.4.2"
pypdf = "^3.16.2"
yt-dlp = "^2023.9.24"
pydub = "^0.25.1"
lark = "^1.1.7"
scikit-learn = "^1.3.1"
instructorembedding = "^1.0.1"
torch = "^2.1.0"
sentence-transformers = "^2.2.2"
matplotlib = "^3.8.0"
panel = "^1.2.3"
docarray = "^0.39.0"
[tool.poetry.group.docs.dependencies]
autodoc_pydantic = "^1.8.0"
myst_parser = "^0.18.1"
nbsphinx = "^0.8.9"
sphinx = "^4.5.0"
sphinx-autobuild = "^2021.3.14"
sphinx_book_theme = "^0.3.3"
sphinx_rtd_theme = "^1.0.0"
sphinx-typlog-theme = "^0.8.0"
sphinx-panels = "^0.6.0"
toml = "^0.10.2"
myst-nb = "^0.17.1"
linkchecker = "^10.2.1"
sphinx-copybutton = "^0.5.1"
nbdoc = "^0.0.82"
[tool.poetry.group.codespell.dependencies]
codespell = "^2.2.0"
[tool.codespell]
skip = '.git,*.pdf,*.svg,*.pdf,*.yaml,*.ipynb,poetry.lock,*.min.js,*.css,package-lock.json,example_data,_dist,examples'
ignore-regex = '.*(Stati Uniti|Tense=Pres).*'
ignore-words-list = 'momento,collison,ned,foor,reworkd,parth,whats,aapply,mysogyny,unsecure,damon,crate'
Here is the error message:
AttributeError: module 'tensorflow' has no attribute 'Tensor'
(hard to believe this error could occur without being detected by the developers, since the tensorflow
module is imported either directly or indirectly by the docarray
module)
and the full stack trace:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[43], line 1
----> 1 cb = cbfs()
3 file_input = pn.widgets.FileInput(accept='.pdf')
4 button_load = pn.widgets.Button(name="Load DB", button_type='primary')
Cell In[42], line 14, in cbfs.__init__(self, **params)
12 self.panels = []
13 self.loaded_file = "docs/MachineLearning-Lecture01.pdf"
---> 14 self.qa = load_db(self.loaded_file,"stuff", 4)
Cell In[41], line 11, in load_db(file, chain_type, k)
9 embeddings = OpenAIEmbeddings()
10 # create vector database from data
---> 11 db = DocArrayInMemorySearch.from_documents(docs, embeddings)
12 # define retriever
13 retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": k})
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/langchain/vectorstores/base.py:417, in VectorStore.from_documents(cls, documents, embedding, **kwargs)
415 texts = [d.page_content for d in documents]
416 metadatas = [d.metadata for d in documents]
--> 417 return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/langchain/vectorstores/docarray/in_memory.py:67, in DocArrayInMemorySearch.from_texts(cls, texts, embedding, metadatas, **kwargs)
45 @classmethod
46 def from_texts(
47 cls,
(...)
51 **kwargs: Any,
52 ) -> DocArrayInMemorySearch:
53 """Create an DocArrayInMemorySearch store and insert data.
54
55 Args:
(...)
65 DocArrayInMemorySearch Vector Store
66 """
---> 67 store = cls.from_params(embedding, **kwargs)
68 store.add_texts(texts=texts, metadatas=metadatas)
69 return store
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/langchain/vectorstores/docarray/in_memory.py:38, in DocArrayInMemorySearch.from_params(cls, embedding, metric, **kwargs)
20 @classmethod
21 def from_params(
22 cls,
(...)
27 **kwargs: Any,
28 ) -> DocArrayInMemorySearch:
29 """Initialize DocArrayInMemorySearch store.
30
31 Args:
(...)
36 **kwargs: Other keyword arguments to be passed to the get_doc_cls method.
37 """
---> 38 _check_docarray_import()
39 from docarray.index import InMemoryExactNNIndex
41 doc_cls = cls._get_doc_cls(space=metric, **kwargs)
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/langchain/vectorstores/docarray/base.py:19, in _check_docarray_import()
17 def _check_docarray_import() -> None:
18 try:
---> 19 import docarray
21 da_version = docarray.__version__.split(".")
22 if int(da_version[0]) == 0 and int(da_version[1]) <= 31:
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/__init__.py:5
1 __version__ = '0.39.0'
3 import logging
----> 5 from docarray.array import DocList, DocVec
6 from docarray.base_doc.doc import BaseDoc
7 from docarray.utils._internal.misc import _get_path_from_docarray_root_level
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/array/__init__.py:1
----> 1 from docarray.array.any_array import AnyDocArray
2 from docarray.array.doc_list.doc_list import DocList
3 from docarray.array.doc_vec.doc_vec import DocVec
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/array/any_array.py:22
3 from typing import (
4 TYPE_CHECKING,
5 Any,
(...)
17 overload,
18 )
20 import numpy as np
---> 22 from docarray.base_doc.doc import BaseDocWithoutId
23 from docarray.display.document_array_summary import DocArraySummary
24 from docarray.exceptions.exceptions import UnusableObjectError
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/base_doc/__init__.py:1
----> 1 from docarray.base_doc.any_doc import AnyDoc
2 from docarray.base_doc.base_node import BaseNode
3 from docarray.base_doc.doc import BaseDoc
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/base_doc/any_doc.py:5
1 from typing import Type
3 from docarray.utils._internal.pydantic import is_pydantic_v2
----> 5 from .doc import BaseDoc
8 class AnyDoc(BaseDoc):
9 """
10 AnyDoc is a Document that is not tied to any schema
11 """
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/base_doc/doc.py:36
34 from docarray.base_doc.base_node import BaseNode
35 from docarray.base_doc.io.json import orjson_dumps_and_decode
---> 36 from docarray.base_doc.mixins import IOMixin, UpdateMixin
37 from docarray.typing import ID
38 from docarray.typing.tensor.abstract_tensor import AbstractTensor
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/base_doc/mixins/__init__.py:1
----> 1 from docarray.base_doc.mixins.io import IOMixin
2 from docarray.base_doc.mixins.update import UpdateMixin
4 __all__ = ['IOMixin', 'UpdateMixin']
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/base_doc/mixins/io.py:25
22 from typing_inspect import get_args, is_union_type
24 from docarray.base_doc.base_node import BaseNode
---> 25 from docarray.typing import NdArray
26 from docarray.typing.proto_register import _PROTO_TYPE_NAME_TO_CLASS
27 from docarray.utils._internal._typing import safe_issubclass
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/__init__.py:3
1 from typing_extensions import TYPE_CHECKING
----> 3 from docarray.typing.bytes import AudioBytes, ImageBytes, VideoBytes
4 from docarray.typing.id import ID
5 from docarray.typing.tensor import ImageNdArray, ImageTensor
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/bytes/__init__.py:1
----> 1 from docarray.typing.bytes.audio_bytes import AudioBytes
2 from docarray.typing.bytes.image_bytes import ImageBytes
3 from docarray.typing.bytes.video_bytes import VideoBytes
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/bytes/audio_bytes.py:9
7 from docarray.typing.bytes.base_bytes import BaseBytes
8 from docarray.typing.proto_register import _register_proto
----> 9 from docarray.typing.tensor.audio import AudioNdArray
10 from docarray.utils._internal.misc import import_library
12 T = TypeVar('T', bound='AudioBytes')
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/__init__.py:5
1 import types
3 from typing_extensions import TYPE_CHECKING
----> 5 from docarray.typing.tensor.audio import AudioNdArray
6 from docarray.typing.tensor.embedding import AnyEmbedding, NdArrayEmbedding
7 from docarray.typing.tensor.image import ImageNdArray, ImageTensor
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/audio/__init__.py:4
1 import types
2 from typing import TYPE_CHECKING
----> 4 from docarray.typing.tensor.audio.audio_ndarray import AudioNdArray
5 from docarray.typing.tensor.audio.audio_tensor import AudioTensor
6 from docarray.utils._internal.misc import (
7 _get_path_from_docarray_root_level,
8 import_library,
9 )
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/audio/audio_ndarray.py:3
1 from docarray.typing.proto_register import _register_proto
2 from docarray.typing.tensor.audio.abstract_audio_tensor import AbstractAudioTensor
----> 3 from docarray.typing.tensor.ndarray import NdArray
6 @_register_proto(proto_type_name='audio_ndarray')
7 class AudioNdArray(AbstractAudioTensor, NdArray):
8 """
9 Subclass of [`NdArray`][docarray.typing.NdArray], to represent an audio tensor.
10 Adds audio-specific features to the tensor.
(...)
50 ---
51 """
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/ndarray.py:31
28 if tf_available:
29 import tensorflow as tf # type: ignore
---> 31 from docarray.typing.tensor.tensorflow_tensor import TensorFlowTensor # noqa: F401
33 if TYPE_CHECKING:
35 from docarray.computation.numpy_backend import NumpyCompBackend
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/tensorflow_tensor.py:34
31 T = TypeVar('T', bound='TensorFlowTensor')
32 ShapeT = TypeVar('ShapeT')
---> 34 tf_base: type = type(tf.Tensor)
35 node_base: type = type(BaseNode)
38 # the mypy error suppression below should not be necessary anymore once the following
39 # is released in mypy: https://github.com/python/mypy/pull/14135
AttributeError: module 'tensorflow' has no attribute 'Tensor'
Example Code
The error occurs in the constructor of class cbfs
, more specifically, when calling DocArrayInMemorySearch.from_documents
in method load_db(...)
:
import panel as pn
import param
def load_db(file, chain_type, k):
# load documents
loader = PyPDFLoader(file)
documents = loader.load()
# split documents
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
docs = text_splitter.split_documents(documents)
# define embedding
embeddings = OpenAIEmbeddings()
# create vector database from data
print("hello 3")
db = DocArrayInMemorySearch.from_documents(docs, embeddings) # ERROR OCCURS <<<<<
# define retriever
print("hello 2")
retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": k})
class cbfs(param.Parameterized):
chat_history = param.List([])
answer = param.String("")
db_query = param.String("")
db_response = param.List([])
def __init__(self, **params):
super(cbfs, self).__init__( **params)
self.panels = []
self.loaded_file = "docs/MachineLearning-Lecture01.pdf"
self.qa = load_db(self.loaded_file,"stuff", 4)
### Python, DocArray & OS Version
```Text
DocArray: 0.39.0
Python: 3.10.10
Affected Components
Here is some more information.
In tensorflow is imported, here is what help(tf)
generates:
Help on package tensorflow:
NAME
tensorflow
PACKAGE CONTENTS
configure
tensorflow (package)
third_party (package)
FILE
(built-in)
I have no idea what was actually imported via import tensorflow as tf
, which occurs in method>
File ~/src/2023/deepleaarning.ai/.venv/lib/python3.10/site-packages/docarray/typing/tensor/ndarray.py:31
28 if tf_available:
29 import tensorflow as tf # type: ignore
---> 31 from docarray.typing.tensor.tensorflow_tensor import TensorFlowTensor # noqa: F401
33 if TYPE_CHECKING:
35 from docarray.computation.numpy_backend import NumpyCompBackend
The error is clearly in typing
somewhere. Inside ndarray:
:
tf_available = is_tf_available()
if tf_available:
import tensorflow as tf # type: ignore
from docarray.typing.tensor.tensorflow_tensor import TensorFlowTensor # noqa: F401
The command import ensorflow as tf
is executed, but the full tensorflow is not imported. No idea why not.
It seems to me that some type-checking could be bypassed to avoid this issue. Thanks.
have u tried installing docarray[tensorflow]
?
I did not. I just installed docarray into poetry. I'll try it. In the meantime, I switched to streamlit which has its own set of issues.
I think this must be related to something in your system.