kyegomez / tree-of-thoughts

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

Home Page:https://discord.gg/qUtxnK2NMf

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Clarity Needed on Claims Made by PrincetonNLP 'Tree of Thoughts' Author

FoobarProtocol opened this issue · comments

I left an issue before that was closed and after careful consideration, you were correct to do so. I apologize for coming out of the gate making claims that the repo was fraudulent. However, I am requesting clarity regarding the claims that have been made against this repo by 'Princeton LLP' here: https://github.com/princeton-nlp/tree-of-thought-llm

Here is what they wrote in relation to your repo:

image

Rather than accepting the comments from the author of the other repo as tacit fact, I wanted to ask for some clarity with respect to the following questions:

  1. Why is the author of the study advocating for people to 'unstar' your GitHub repo? (more on this later, I do not necessarily agree with the tactics of the other author).

  2. What is your stance on what has been said about this repo by one of the authors of the 'Tree of Thought' methodology? Do you find these comments to be valid? Why or why not?

  3. When considering the fact that there are more implementations of the 'Tree of Thought' mechanism apart from just his, what do you think the motivation here is for somebody to insist that nobody should be adhering to your repo?

  4. The author of the study suggested that all of your repos are crafted using ChatGPT, what is your response to this?

Clarifying My Position

After taking time to go through your README.md and the accompanying Python code, I find it hard to imagine that this is a BS implementation of the 'Tree of Thoughts' prompt strategy. Your codebase is more coherent and 'logical' than the princeton-nlp implementation.

Even if what the author of that repo stated was true regarding the supposed inaccuracies in your implementation of the Tree of Thoughts' prompt strategy, one would think that the they'd consider it more constructive to make a pull request that rectifies the alleged deficits. Instead, they elected to put you on blast for it, then advocate that no one else use or implement your iteration of the 'Tree of Thoughts' guide.

Also, the author of the princeton-nlp repo failed to identify exactly what issues existed within the repo that would've led to an inaccurate / degraded prompt output. I find that to be problematic as well.

Purpose of This Issue

To give you a chance to address these issues if you have the chance. I know that I'm not the only person that's looked up whether there are any 'Tree of Thoughts' repos out there and stumbled across both yours and the one from princeton-nlp. After reading the princeton-nlp disclaimer in their README.md, I was a bit hesitant at first due to the conviction their claims appear to have.

However, it is not fair for them to make such accusations without any proof. I believe in the principles behind 'open source' and, in my opinion, that researcher violated them.

With that being said, I am going to take some time soon to spin up the code here and see what results I get. My impression after reading the paper is that this methodology is more of a conceptual idea vs. a concrete, finite implementation that can only be iterated with specific code. So it confuses me as to why the author of the princeton-nlp 'Tree of Thoughts' repo would take such an aggressive stance.

Why I'm Asking for Clarity

To be clear, I'm not asking you to 'defend yourself' per se, but I do think that clarity is warranted for the sake of the broader open source community. If the author of the 'opposing' 'Tree of Thoughts' repo is incorrect in their assertions about your repo, then I believe that this is a fact that should be enumerated. Its a disservice to the community at large if developers and programmers are being steered away from a legitimate implementation of this repo. So my hopes are that greater proof will assist the open source community with parsing between the two repos & determining 'what's what'.

Bumping this for a comment once again. Assuming that the claims made by the other repository author are unequivocally false, I could see why you would be reluctant to answer this issue or take the time to address this matter. Likely the thinking here is that users can simply verify the code themselves if they are in any doubt/question.

If this is the case, then could you explain how one would go about doing so? I'd be more happy to put together the tests myself and then add into this repo as a pull request for you to integrate at a latter point in time. I'd also make sure to comprehensively document my process so that other users could replicate it themselves if they felt so inclined.

I'm always more willing to help than encumber.

Hey, I'm thankful for your open mindedness and patience.

  • First, They release a paper with no code or reproducible experiment, this I deem as heretical to science and so I implement it and it's popular really quick because this technique is truly powerful.
  • They get mad that my repo and code is better than their's and they published they paper, they feel entitled even though I reproduced the entire paper based on 4 phrases, dfs, bfs (search algos), generate solutions, and generate thoughts and this is it. I didn't even read the full paper when I first started to implement it.
  • The reason they want people to unstar my repo is because they are jealous that they made a mistake by not sharing the code when they published the paper as real scientists would do. If you do not publish your code as a AI research scientists you are a heretic, as your work cannot be tried and tested.
  • and the code works amazingly much better than theirs, I looked at their code and couldn't figure out how to run it for hours, as well as other people have reported the same.
  • the motivations are jealously, self hatred, guilt, envy, inferiority complex, ego, and much more psychographic principles.
  • And, what proof do they have that my code is chatgpt generated? I mean look at the number of commits I have garned in just half a year, 3000+ commits, with hundreds a day. They have 0 proof of this and they are just trying to discredit me and our work.
  • I haven't answered because I am working on 40+ projects all at once

You the man. Screw the haters.

Crypto bros at it again lmao.

Maybe try getting an actual job instead of making commits where you change a single word each.

image

This is just too hilarious. I'm sure you're gonna be the savior of humanity.

Crypto bros at it again lmao.

Maybe try getting an actual job instead of making commits where you change a single word each.

So I'm not sure if you just finished your crack binge, but I'm busy working on a really high level project that involves prompt engineering as well as actually fine-tuning a large language model from scratch (the pre-trained model state). Not only that, I'm pretty much swapping out all of the different constituent pieces of the LLM itself with custom built alternatives.

I've gained expertise on countless machine-learning concepts like supervised and unsupervised training, reinforcement learning, loss functions, optimization methods, regularization techniques, cross-validation, overfitting and underfitting, bias-variance tradeoff and ensemble methods. I also am highly informed when it comes to perceptrons, multilayer perceptrons, activation functions, backpropagation, gradient descent, stochastic gradient descent, batch normalization, dropout rates, convolutional neural networks, recurrent neural networks (LSTM & GRUs), attention mechanisms, and transformers. Of course, I'd be remiss if I declined to mention my familiarity with various embedding methodologies like word2vec, GloVe, fastText etc., or concepts under this umbrella within the scope of language models such as embedding matrices or context-based vs frequency-based embeddings. Beyond that I have extensive knowledge of n-gram models, feedforward and recurrent neural language models and autoregressive models like GPT. I am also well-versed in data preprocessing steps like: cleaning, normalization, handling rare words, padding and masking and fine-tuning/training concepts like evaluation metrics, learning rates, batch sizes, epochs, distributed training, mixed precision, gradient accumulation, multitask learning, memory augmented networks and adversarial training.

These notes are just a microcosm of what I do on an average day:

image

Make sure you do your homework on who the fuck you're speaking to before you even dare to speak on my name next time, child. You really got me fucked up. My genius starts where your intelligence drops off at. I never needed a teacher, mentor, class, lecture, or degree to do what I'm doing. I'm naturally gifted. I do every single thing you've ever attempted to do on planet earth better than you - crypto included.

I'm a multi-faceted, legendary individual that's trail blazed in an industry that's traditionally devoid of individuals from my demographic. I'm literally breaking barriers and making history with the shit that I'm working on.

Now what have you done lately? I'll wait.

Dumbass.

This is just a microcosm of the work that I do

finally.mp4

Talking about "crypto bros at it again" - who is a crypto bro here? You're some fucking nobody that's never done or contributed shit to anything on the face of planet earth assuming that I'm a "crypto bro" because you saw me star a few repositories related to the space.

I work harder than you, I'm smarter than you, I've done more than you, I'm more talented and capable than you'll ever be at any point in your life up to this point and including until you die and you really can't touch me in any area, facet or aspect of development, code, software engineering, computers or technology itself. The same applies to any other industry or field in existence. I'm fundamentally better than you at everything. You would do anything in life to be me.

But you can't. That's why you had nothing to contribute to this thread of conversation. You probably don't even understand the subject matter being discussed here. I doubt you have the ability to parse the nuance in the dialogue that I'm having here with the developer of this repo. I'm almost certain that you've never built shit yourself. You're another spreadsheet monkey with a subscription to Khan Academy praying to learn enough about "basic coding" to finally get your Fiverr page started.

You should be humbled by the fact I even took the time to respond to something this asinine. As you can see from what I just posted off top, my time is valuable. So do me a favor and don't ever even consider talking to me or responding to anything that I ever write on any social media platform ever again unless you contribute something that at least meets 75% of the quality of output that I do.

Until you can do so, sit the fuck down and shut up like a good boy.