'NoneType' object has no attribute 'squeeze'
xunfeng1980 opened this issue · comments
Issue Description
When use #1 (comment) link on VS Code,the following error occurs
fauxpilot-copilot_proxy-1 | INFO: 2023-02-21 06:44:05,739 :: 10.1.41.148:36532 - "POST /v1/engines/codegen/completions HTTP/1.1" 500 Internal Server Error
fauxpilot-copilot_proxy-1 | ERROR: Exception in ASGI application
fauxpilot-copilot_proxy-1 | Traceback (most recent call last):
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
fauxpilot-copilot_proxy-1 | result = await app( # type: ignore[func-returns-value]
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
fauxpilot-copilot_proxy-1 | return await self.app(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
fauxpilot-copilot_proxy-1 | await super().__call__(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
fauxpilot-copilot_proxy-1 | await self.middleware_stack(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
fauxpilot-copilot_proxy-1 | raise exc
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
fauxpilot-copilot_proxy-1 | await self.app(scope, receive, _send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
fauxpilot-copilot_proxy-1 | raise exc
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
fauxpilot-copilot_proxy-1 | await self.app(scope, receive, sender)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
fauxpilot-copilot_proxy-1 | raise e
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
fauxpilot-copilot_proxy-1 | await self.app(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
fauxpilot-copilot_proxy-1 | await route.handle(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
fauxpilot-copilot_proxy-1 | await self.app(scope, receive, send)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
fauxpilot-copilot_proxy-1 | response = await func(request)
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 231, in app
fauxpilot-copilot_proxy-1 | raw_response = await run_endpoint_function(
fauxpilot-copilot_proxy-1 | File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
fauxpilot-copilot_proxy-1 | return await dependant.call(**values)
fauxpilot-copilot_proxy-1 | File "/python-docker/./app.py", line 56, in completions
fauxpilot-copilot_proxy-1 | content = codegen(data=data)
fauxpilot-copilot_proxy-1 | File "/python-docker/./utils/codegen.py", line 248, in __call__
fauxpilot-copilot_proxy-1 | completion, choices = self.generate(data)
fauxpilot-copilot_proxy-1 | File "/python-docker/./utils/codegen.py", line 166, in generate
fauxpilot-copilot_proxy-1 | lp_data = result.as_numpy("output_log_probs").squeeze(1)
fauxpilot-copilot_proxy-1 | AttributeError: 'NoneType' object has no attribute 'squeeze'
fauxpilot-triton-1 | The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Expected Result
How to Reproduce
Further Information
model: py-model
Ubuntu 20.04.2 LTS
CUDA
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.54 Driver Version: 510.54 CUDA Version: 11.6 |
Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!
Hi @xunfeng1980, the python model doesn't support logprobs yet. If you set logprobs option to null, you won't face this issue. I'll keep this open till logprobs is in.