Adapted form ChunkLlama's needle in haystack evals @ https://github.com/HKUNLP/ChunkLlama/tree/main/needle_in_a_haystack to use vLLM.
Adapted form ChunkLlama's needle in haystack evals @ https://github.com/HKUNLP/ChunkLlama/tree/main/needle_in_a_haystack to use vLLM.