https://eugeneyan.com/writing/llm-experiments/
utterances-bot opened this issue · comments
Experimenting with LLMs to Research, Reflect, and Plan
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
"If we use exact nearest neighbours, we would get perfect recall of 1.0 but with higher latency (think seconds)."
Exact search is performant up to tens of thousands of documents / vectors. Is the document store you're embedding really that large?
Not currently. Nonetheless, this is meant to scale to larger and far more documents, such as books and papers (idea here).
where is the source code for that?
Great read, nice work on this. Thanks for highlighting the issues with retrieval. I’ve recently started building an LLM-powered assistant type app and was surprised to see how difficult the retrieval step is. There are lots of blog posts out there about how to set up a vector search tool but very few about how to optimize and troubleshoot queries and embeddings.
where is the source code for that?
Currently private. It’s a mess and I’m embarrassed lol 🙈 Also needs to be scrubbed of credentials.