Giters

jxnl / n-levels-of-rag

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bit of feedback on levels 1 and 2

JungeAlexander opened this issue · comments

commented

Hi @jxnl - first of all, I'm a fan of your work, keep it up!

A bit of feedback to levels 1 and 2 of the current article:

NB: my feedback concerns your blog post but I think it is identical to the repo here.

Level 1

  • in Processing Pipeline, sem seems unused and could be confusing

Level 2

  • when entering Level 2, I suggest you recommend the reader to obtain/label at least some kind of initial evaluation set, otherwise, it is hard to justify that chunking, re-ranking, etc. actually makes things "better" (and by how much, and when to try making them "better, etc.)
    • consider also to pull some sections from level 4 forward
    • or maybe level 2 could become level 4 while 3 becomes 2 and 4 becomes 3. The idea is that level 1 gets you the initial prototype, 2 & 3 set the foundation to make it better, and level 4 actually makes it better building on the right foundation.
  • add an example of why query rewriting/expansion makes sense qualitatively (or quantitatively if you can), e.g. a query like "show me recent work done on topic A and B" ending in "no-man's land" in embedding space (not to mention what "recent" means)