fauxpilot in cuda 11.7
younis-bash opened this issue · comments
Hello
I'm trying to setup fauxpilot in cuda 11.7 but I got this warning
WARNING: CUDA Minor Version Compatibility mode ENABLED. fauxpilot-triton-1 | Using driver version 510.108.03 which has support for CUDA 11.6. This container fauxpilot-triton-1 | was built with CUDA 11.7 and will be run in Minor Version Compatibility mode. fauxpilot-triton-1 | CUDA Forward Compatibility is preferred over Minor Version Compatibility for use fauxpilot-triton-1 | with this container but was unavailable:
otherwise, when I make a request it returns an empty response like this
<OpenAIObject id=cmpl-Ndy932EKwgSk0HR7iyiV3keoqoRF7 at 0x7f07c74b41d0> JSON: { "choices": [], "id": "cmpl-Ndy932EKwgSk0HR7iyiV3keoqoRF7" }
in addition for that, I investigated where it's the error happened actually and I detected it in codegen.py line 157
result = self. client.infer(model_name, inputs)
if you could please give me feedback so I can solve the issue
sincerely
Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!