Giters

nomic-ai / gpt4all

gpt4all: run open-source LLMs anywhere

Home Page:https://gpt4all.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature] Support old MPT GGUF conversions with duplicated output tensor

dlippold opened this issue · comments

commented

Bug Report

The fine-tuned MPT model from https://huggingface.co/maddes8cht/mosaicml-mpt-7b-instruct-gguf/ in quantization Q4_1 was usabel in release 2.7.2 but not longer in 2.7.3 and later. In particular it is currently not usable.

When I try to load the model file I get the following error message:

Could not load model due to invalid model file for mosaicml-mpt-7b-instruct-Q4_1.gguf

The reason of the problem may have to do with #2006

Steps to Reproduce

  1. Download the model file from the specified URL
  2. Start GPT4all
  3. Choose the downloaded model file

Expected Behavior

The model file should be loaded.

Your Environment

  • GPT4All version: 2.7.2, 2.7.3, 2.7.5
  • Operating System: Ubuntu Linux 22.04.
  • Chat model used (if applicable): see above
commented

I fixed this upstream in ggerganov/llama.cpp#6139 which should make it into the next release of GPT4All (already included in #2310).