A lot of errors
jbdatascience opened this issue · comments
When I excute this code:
`# !cat /content/tree-of-thoughts/huggingfaceExample.py
from tree_of_thoughts.treeofthoughts import OpenAILanguageModel, GuidanceOpenAILanguageModel, TreeofThoughts, OptimizedOpenAILanguageModel, OptimizedTreeofThoughts, HuggingLanguageModel
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name="gpt2"
model_tokenizer="gpt2tokenizer"
model = HuggingLanguageModel(model_name, model_tokenizer)
class HuggingLanguageModel:
def init(self, model_name):
self.model = AutoModelForCausalLM.from_pretrained(model_name)
self.tokenizer = AutoTokenizer.from_pretrained(model_name)
def generate_text(self, prompt, max_length=50):
inputs = self.tokenizer(prompt, return_tensors="pt")
outputs = self.model.generate(inputs["input_ids"], max_length=max_length)
generated_text = self.tokenizer.decode(outputs[0], skip_special_tokens=True)
return generated_text
Initialize the HuggingLanguageModel with the GPT-2 model
model_name = "gpt2"
model = HuggingLanguageModel(model_name,
model_tokenizer="gpt2",
verbose=True)
#choose search algorithm('BFS' or 'DFS')
search_algorithm = "DFS" # "BFS"
#cot or propose
strategy="cot"
value or vote
evaluation_strategy = "value"
gpt2_model = HuggingLanguageModel(model_name)
tree_of_thoughts= OptimizedTreeofThoughts(model, search_algorithm)
input_problem = "use 4 numbers and basic arithmetic operations (+-*/) to obtain 24"
k = 5
T = 3
b = 5
vth = 0.5
timeout = 10
confidence = 0.8 #cmodel is confident on performance
max_iterations = 40 #tree branh nodes
convergence_threshold = 0.01
convergence_count = 5
solution = tree_of_thoughts.solve(input_problem, k, T, b, vth, timeout, confidence_threshold=confidence, max_iterations=max_iterations, convergence_threshold=convergence_threshold, convergence_count=convergence_count)
#use the solution in your production environment
print(f"solution: {solution}")`
I get a lot of errors:
"
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Setting pad_token_id to eos_token_id:50256 for open-end generation.
Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens.
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
...
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - * / ) t o o b t a i n 2 4
Error: num_return_sequences has to be 1 when doing greedy search, but is 5.
result: None
Saving the current tree and metrics.
solution: None
"
What could be the problem and how can it be solved?
It ran I believe but the model is just very dumb
It ran I believe but the model is just very dumb
Couldn't we use a better Language Model ? There are a lot of them around these days, their number increasing nearly every day! Yesterday I learned from a model named Falcon: “Falcon 40B,” is now open source for research and commercial use.
Falcon, a fundamental large language model (LLM) trained on one trillion tokens and 40 billion parameters.
Sounds like an Open Source LLM of high quality. Perhaps you could use it in your Tree of Thoughts implementation?
See:
https://www.opensourceforu.com/2023/05/open-source-falcon-40b-large-language-model-gets-released/
Details:
Falcon-40B is a 40B parameters causal decoder-only model built by TII and trained on 1,000B tokens of RefinedWeb enhanced with curated corpora. It is made available under the TII Falcon LLM License - Hugging Face !!!!! !!!!!
Absolutely, you can plug in and play! Just input your own model and tokenizer! I'm here for you if you seek me
Absolutely, you can plug in and play! Just input your own model and tokenizer! I'm here for you if you seek me
I will do that ! But at the moment I am trying to get the large 40B Falcon working. His little brother Falcon 7B already works, so I could give it a try on this Train of Thoughts implementation !
Absolutely! Input the model name and the tokenizer for that model and it should work! Let me know if you need anything