Pip installation failed with conflicting tritonclient error

[Yaxiong Zhao]

Workaround

Removed explicit torchclient deps from pyproject.toml

-    "tritonclient[all] >= 2.38",
+    # "tritonclient[all] >= 2.38",

And install torchclient with pip install 'torchclient[all]'

The error looks like below:

PS: posting from the live session of GTC 2024 (benchmarking and optimizing LLM inference), great release!

[Ryan McCormick]

Hi @yzhao-2023, thanks for filing our first issue! Hope you had a good time at GTC this week!

Can you share the exact steps you followed to reproduce this error and some more details on the environment / currently installed versions?

I wasn't able to reproduce this error by doing the following:

# Use docker container for consistency
docker run -it \
    --name triton \
    --gpus all --network host \
    --shm-size=1g --ulimit memlock=-1 \
    -v /tmp:/tmp \
    -v ${HOME}:/workspace \
    -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
    -w /workspace \
    nvcr.io/nvidia/tritonserver:24.02-trtllm-python-py3

# Pre-install tritonclient before installing CLI
pip install tritonclient[all]

# Install CLI
pip install git+https://github.com/triton-inference-server/triton_cli

So I just wanted to double check the steps you're taking.

[Ryan McCormick]

Closing due to inactivity.

Please try with the latest version and let us know if you run into any issues.