bentoml / OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

Home Page:https://bentoml.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bug: pip package version ssues

vilewired opened this issue · comments

commented

Describe the bug

When installing openllm, it installs:
openllm==0.4.44
openllm-client==0.4.44
openllm-core==0.4.44

and when trying to run a model it will ask you to install openllm[vllm]. But when you install the extra vllm, your openllm downgrades to version openllm==0.1.20, which seems pretty old. I mean since you are deprecating pytorch, then at least vllm should be supported for your later versions.. or have I been missing something during installation?

To reproduce

pip install openllm
// check version
pip install openllm[vllm]
// version downgraded

Logs

No response

Environment

bentoml:

Environment variable

BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information

bentoml: 1.1.11
python: 3.12.3
platform: Linux-6.8.7-arch1-2-x86_64-with-glibc2.39
uid_gid: 1000:1000

pip_packages
accelerate==0.29.3
aiohttp==3.9.5
aiosignal==1.3.1
anyio==4.3.0
appdirs==1.4.4
asgiref==3.8.1
attrs==23.2.0
bentoml==1.1.11
bitsandbytes==0.41.3.post2
build==0.10.0
cattrs==23.1.2
certifi==2024.2.2
charset-normalizer==3.3.2
circus==0.18.0
click==8.1.7
click-option-group==0.5.6
cloudpickle==3.0.0
coloredlogs==15.0.1
contextlib2==21.6.0
cuda-python==12.4.0
datasets==2.19.0
deepmerge==1.1.1
Deprecated==1.2.14
dill==0.3.8
distlib==0.3.8
distro==1.9.0
einops==0.8.0
fastcore==1.5.32
filelock==3.14.0
filetype==1.2.0
frozenlist==1.4.1
fs==2.4.16
fsspec==2024.3.1
ghapi==1.0.5
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
huggingface-hub==0.22.2
humanfriendly==10.0
idna==3.7
importlib-metadata==6.11.0
inflection==0.5.1
Jinja2==3.1.3
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
mypy-extensions==1.0.0
networkx==3.3
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-ml-py==11.525.150
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.1.105
openllm==0.4.44
openllm-client==0.4.44
openllm-core==0.4.44
opentelemetry-api==1.20.0
opentelemetry-instrumentation==0.41b0
opentelemetry-instrumentation-aiohttp-client==0.41b0
opentelemetry-instrumentation-asgi==0.41b0
opentelemetry-sdk==1.20.0
opentelemetry-semantic-conventions==0.41b0
opentelemetry-util-http==0.41b0
optimum==1.19.1
orjson==3.10.1
packaging==24.0
pandas==2.2.2
pathspec==0.12.1
pillow==10.3.0
pip-requirements-parser==32.0.1
pip-tools==7.3.0
platformdirs==4.2.1
prometheus_client==0.20.0
protobuf==5.26.1
psutil==5.9.8
pyarrow==16.0.0
pyarrow-hotfix==0.6
pydantic==1.10.15
Pygments==2.17.2
pyparsing==3.1.2
pyproject_hooks==1.1.0
python-dateutil==2.9.0.post0
python-json-logger==2.0.7
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
pyzmq==26.0.2
regex==2024.4.28
requests==2.31.0
rich==13.7.1
safetensors==0.4.3
schema==0.7.5
scipy==1.13.0
sentencepiece==0.2.0
setuptools==69.5.1
simple-di==0.1.5
six==1.16.0
sniffio==1.3.1
starlette==0.37.2
sympy==1.12
tokenizers==0.19.1
torch==2.3.0
tornado==6.4
tqdm==4.66.2
transformers==4.40.1
typing_extensions==4.11.0
tzdata==2024.1
urllib3==2.2.1
uvicorn==0.29.0
virtualenv==20.26.1
watchfiles==0.21.0
wheel==0.43.0
wrapt==1.16.0
xxhash==3.4.1
yarl==1.9.4
zipp==3.18.1

transformers-cli:

ransformers-cli env
Traceback (most recent call last):
File "/home/iam/.local/bin/transformers-cli", line 5, in
from transformers.commands.transformers_cli import main
ModuleNotFoundError: No module named 'transformers'

System information (Optional)

OS: Arch Linux