zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks

Home Page:https://docs.privategpt.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bulk Local Injestion: How to initiate from Dockerfile? & "--watch" is not working

K-J-VV opened this issue · comments

I'm running PGPT in docker and following the documentation here to create an "injestion" folder: https://docs.privategpt.dev/manual/document-management/ingestion

After I run the container I able to get the following command to work (i.e., PGPT checks the folder and embeds the documents)

make ingest /home/worker/app/injest -- --watch --log-file /home/worker/app/injest/.PrivateGPT.log
Ideally I'd just like this to be set up before the container runs. I can't seem to figure out how to modify the Dockerfile to have this run when the container initially starts.

Also, I've noticed the --watch flag doesn't seem to work. I have to re-run the entire command to have PGPT re-scan the folder

However, despite it not actually embedding the new files based on the log it looks like the error log is constantly detecting changes?

[2024-05-06 11:44:33.883] [INFO] Detected change in at path=/home/worker/app/injest/.PrivateGPT.log, ingesting
[2024-05-06 11:44:33.884] [INFO] Started ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:34.359] [INFO] Completed ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:34.359] [INFO] Detected change in at path=/home/worker/app/injest/.PrivateGPT.log, ingesting
[2024-05-06 11:44:34.359] [INFO] Started ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:34.911] [INFO] Completed ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:34.911] [INFO] Detected change in at path=/home/worker/app/injest/.PrivateGPT.log, ingesting
[2024-05-06 11:44:34.911] [INFO] Started ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:35.382] [INFO] Completed ingesting file=/home/worker/app/injest/.PrivateGPT.log
[2024-05-06 11:44:35.383] [INFO] Detected change in at path=/home/worker/app/injest/.PrivateGPT.log, ingesting
[2024-05-06 11:44:35.383] [INFO] Started ingesting file=/home/worker/app/injest/.PrivateGPT.log

FYI, I'm only able to get the command make ingest to work when I run it as root & after I copy both Makefile and pyproject.toml into the /home/worker/app directory. I can't seem to figure out how to run this command "detached"