where is mycode error?
yutaolife opened this issue · comments
there are not any output. where is mycode error?
import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, set_seed
tokenizer = AutoTokenizer.from_pretrained("/Users/yutao/Documents/MyCode/meditron_7b/model")
model = AutoModelForCausalLM.from_pretrained("/Users/yutao/Documents/MyCode/meditron_7b/model")
question = "What to do about high blood pressure?"
inputs = tokenizer(question, return_tensors="pt")
set_seed(42)
answers = []
with torch.inference_mode():
beam_output = model.generate(**inputs,
max_new_tokens=1024,
num_beams=1,
pad_token_id=2,
eos_token_id=2,
early_stopping=False,
do_sample=False,
)
answers.append(tokenizer.decode(beam_output[0], skip_special_tokens=True))
print("answers: "+answers)
Hi. Can you try LM Studio ? It works so easily.
Except if you need to chain the output