epfLLM / meditron

Meditron is a suite of open-source medical Large Language Models (LLMs).

Home Page:https://huggingface.co/epfl-llm

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

eval generation path issue

sia-cerebras opened this issue · comments

Hello, I'm trying to use your eval pipeline.
I ran ./inference_pipeline.sh -b pubmedqa -c gpt2 -s 0 -m 0 -out_dir out_dir
After it is done with generation I get the following error:

Stored pubmedqa generations to the following path: ../benchmarks/generations/pubmedqa-gpt2.jsonl
Traceback (most recent call last):
  File "evaluate.py", line 475, in <module>
    main(args)
  File "evaluate.py", line 390, in main
    data = load_jsonl(path)
  File "evaluate.py", line 39, in load_jsonl
    with open(filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '../benchmarks/generations/pubmedqa/pubmedqa-gpt2.jsonl'

It looks like the generations are saved in: meditron/benchmarks/generations/pubmedqa-gpt2.jsonl
But eval looks for them in: meditron/benchmarks/generations/pubmedqa/pubmedqa-gpt2.jsonl'