RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
2579356425 opened this issue · comments
I get error RuntimeError: probability tensor contains either inf, nan or element < 0 when infering with gemma-7b-quant.ckpt. The environments are as follows:
fairscale==0.4.13
filelock==3.9.0
fsspec==2023.4.0
gemma==0.1
immutabledict==4.1.0
Jinja2==3.1.2
MarkupSafe==2.1.3
mpmath==1.3.0
networkx==3.2.1
numpy==1.24.4
nvidia-cublas-cu11==11.11.3.6
nvidia-cuda-cupti-cu11==11.8.87
nvidia-cuda-nvrtc-cu11==11.8.89
nvidia-cuda-runtime-cu11==11.8.89
nvidia-cudnn-cu11==8.7.0.84
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.3.0.86
nvidia-cusolver-cu11==11.4.1.48
nvidia-cusparse-cu11==11.7.5.86
nvidia-nccl-cu11==2.19.3
nvidia-nvtx-cu11==11.8.86
pillow==10.2.0
sentencepiece==0.1.99
sympy==1.12
torch==2.2.1+cu118
torchaudio==2.2.1+cu118
torchvision==0.17.1+cu118
triton==2.2.0
typing_extensions==4.8.0
me too， did you solve it?
me too， did you solve it?
I set model_config.quant = True in run.py. The problem is solved. But the answer is wrong.
it works ! but it’s answer is very rude .
Duplicate issue #10.
The same problem i meet. anybody who knows why?