epfLLM / meditron

Meditron is a suite of open-source medical Large Language Models (LLMs).

Home Page:https://huggingface.co/epfl-llm

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

where is mycode error?

yutaolife opened this issue · comments

there are not any output. where is mycode error?
import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, set_seed

tokenizer = AutoTokenizer.from_pretrained("/Users/yutao/Documents/MyCode/meditron_7b/model")
model = AutoModelForCausalLM.from_pretrained("/Users/yutao/Documents/MyCode/meditron_7b/model")

question = "What to do about high blood pressure?"
inputs = tokenizer(question, return_tensors="pt")
set_seed(42)

answers = []

with torch.inference_mode():
beam_output = model.generate(**inputs,
max_new_tokens=1024,
num_beams=1,
pad_token_id=2,
eos_token_id=2,
early_stopping=False,
do_sample=False,
)
answers.append(tokenizer.decode(beam_output[0], skip_special_tokens=True))

print("answers: "+answers)

commented

Hi. Can you try LM Studio ? It works so easily.
Except if you need to chain the output