Uploading model to HF

[Vineeth Venugopal]

HI All, I am having some trouble uploading a fine tuned model to the hugging face hub. When I do the following:

  'repo_name',
  'model_path',
  repo_type='model',
  private=False,
  commit_message='Upload trained [Ludwig](https://ludwig.ai/latest/) model weights',
  commit_description=None
)```
I get the following error:
```File ~/anaconda3/envs/vineeth_12/lib/python3.10/site-packages/ludwig/utils/upload_utils.py:107, in BaseModelUpload._validate_upload_parameters(repo_id, model_path, repo_type, private, commit_message, commit_description)
    105 trained_model_artifacts_path = os.path.join(model_path, "model", "model_weights")
    106 if not os.path.exists(trained_model_artifacts_path):
--> 107     raise Exception(
    108         f"Model artifacts not found at {trained_model_artifacts_path}. "
    109         f"It is possible that model at '{model_path}' hasn't been trained yet, or something went"
    110         "wrong during training where the model's weights were not saved."
    111     )
    113 # Make sure the model's saved artifacts either contain:
    114 # 1. pytorch_model.bin -> regular model training, such as ECD or for LLMs
    115 # 2. adapter_model.bin -> LLM fine-tuning using PEFT
    116 files = set(os.listdir(trained_model_artifacts_path))

Exception: Model artifacts not found at finetuned_10epoch_MatKG_Llama2_7b_4bit/model/model_weights. It is possible that model at 'finetuned_10epoch_MatKG_Llama2_7b_4bit' hasn't been trained yet, or something wentwrong during training where the model's weights were not saved.

Whn I checked, I found that the model path that I created by model.save('model_path') does not have a 'model' subfolder. This was resolved by manually creating a 'model' folder inside model_path and then copying all the files into this folder.

However, now I am getting the error:

HfHubHTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/repos/create (Request ID: Root=1-6627c70a-1fcc7ba711bc8a6172dc50f1;dd8d6ed4-47b5-4d01-a8b6-ab9a3da0653b)

Invalid username or password.

Any suggestions on how to resolve this?

Environment (please complete the following information):

• OS: [e.g. iOS]
• Version [e.g. 22]
• 3.9
• 0.10.3

[Vineeth Venugopal]

I was able to solve this problem in two ways:

(1) logging out and logging in again:

import huggingface_hub
huggingface_hub.logout()

(2) I loaded the model files one by one rather than as one folder.

 for file in [ 'adapter_model.bin', 'adapter_config.json', 'adapter_model.safetensors']:
        path = os.path.join(base_path, 'model', 'model_weights', file)
           api.upload_file(
                        path_or_fileobj= path,
                        path_in_repo = file,
                        repo_id=repo_id,
                        repo_type='model',
                      commit_message=commit_message,
                      commit_description=commit_message,
                    )