Unable to Load Pretrained Model Due to Missing config.json

[Lauren Vega]

Issue Description:
I encountered an issue while attempting to load a pretrained model using the provided code snippet. It seems that the error is related to the absence of the config.json file in the model directory. I'm seeking guidance on how to properly load the model and perform inference on a Linux system.
Deployment Steps:

Create a conda environment and install required packages:

conda create -n doc python=3.8
conda activate doc
pip install numpy torch datasets huggingface_hub transformers trl bitsandbytes sentencepiece openai tvm peft onnx jupyter
git clone https://github.com/llSourcell/DoctorGPT.git

Download the pretrained model from https://huggingface.co/llSourcell/medllama2_7b and save it locally.
Attempt to debug using the provided code snippet:

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    HfArgumentParser,
    AutoTokenizer,
    TrainingArguments,
)

local_model_path = "/data/gitclone/text-generation-webui/models/llSourcell_medllama2_7b"
model = AutoModelForCausalLM.from_pretrained(local_model_path)

Error Details:
The error arises because the config.json file is missing in the model directory.

Request for Assistance:
Could you please provide guidance on the correct approach to load the pretrained model without encountering the missing config.json error? Additionally, I would appreciate instructions on how to perform inference using the loaded model on a Linux system.

Error Screenshot:

Thank you for your help!

[zhuanghuayan]

My problem is the same, I hope the author solves it

[Lauren Vega]

This issue can be resolved by downloading the latest model that includes the config.json file.
Linux, Windows:
https://huggingface.co/llSourcell/medllama2_7b
Android, IOS:
https://huggingface.co/llSourcell/doctorGPT_mini