tasks must not change working directory to avoid "No such file or directory: '.doit.db.dat'"
goretkin opened this issue · comments
Running the dodo.py
below:
import os
def task_hello():
"""hello"""
def python_hello(targets):
with open(targets[0], "a") as output:
output.write("Python says Hello World!!!\n")
os.chdir("..")
return {
'actions': [python_hello],
'targets': ["hello.txt"],
}
def task_goodbye():
"""goodbye"""
def python_goodbye(targets):
with open(targets[0], "a") as output:
output.write("Python says Goodbye World!!!\n")
return {
'actions': [python_goodbye],
'file_dep': ["hello.txt"],
'targets': ["goodbye.txt"],
}
if __name__ == '__main__':
import doit
doit.run(globals())
Gives the following output:
changing directory causes this issue:
. hello
. goodbye
DependencyError - taskid:goodbye
ERROR: Task 'goodbye' saving success: Dependent file 'hello.txt' does not exist
Traceback (most recent call last):
File "[...]\python_environment\lib\site-packages\doit\doit_cmd.py", line 295, in run
return command.parse_execute(args)
File "[...]\python_environment\lib\site-packages\doit\cmd_base.py", line 151, in parse_execute
return self.execute(params, args)
File "[...]\python_environment\lib\site-packages\doit\cmd_base.py", line 605, in execute
return self._execute(**exec_params)
File "[...]\python_environment\lib\site-packages\doit\cmd_run.py", line 265, in _execute
return runner.run_all(self.control.task_dispatcher())
File "[...]\python_environment\lib\site-packages\doit\runner.py", line 260, in run_all
self.finish()
File "[...]\python_environment\lib\site-packages\doit\runner.py", line 239, in finish
self.dep_manager.close()
File "[...]\python_environment\lib\site-packages\doit\dependency.py", line 522, in close
self.backend.dump()
File "[...]\python_environment\lib\site-packages\doit\dependency.py", line 179, in dump
self._dbm[task_id] = self.codec.encode(self._db[task_id])
File "[...]\contrib\Python37\lib\dbm\dumb.py", line 208, in __setitem__
self._addkey(key, self._addval(val))
File "[...]\contrib\Python37\lib\dbm\dumb.py", line 164, in _addval
with _io.open(self._datfile, 'rb+') as f:
FileNotFoundError: [Errno 2] No such file or directory: '.doit.db.dat'
What if the tasks themselves took care of restoring the working directory, since doit
relies on it?
Upvote & Fund
- We're using Polar.sh so you can upvote and help fund this issue.
- We receive the funding once the issue is completed & confirmed by you.
- Thank you in advance for helping prioritize & fund our backlog.
what if you have 2 different tasks running in different threads?
@schettino72 , I am not sure I understand. Is that an argument against doing any process-wide state changes, like changing the current directory, within a task?
Is that an argument against doing any process-wide state changes, like changing the current directory, within a task?
yes. of course you could do this on your own code on your own risk. but doit
will not support that.
so what then, is the standard way of operating in a subdirectory of the root (which has dodo.py).
E.g. I want to do a docker compose build
in my deployment subdirectory. Manually, from the root, this is:
cd ./deployment
docker compose build
I'm sure I could do it without the cd using docker command-line args, but many similar tasks are neater when working from a specific cwd.
What's the doit way of operating in a subdir?
(I've scoured the docs for cd, directory, chdir, and can't find much here)
What's the doit way of operating in a subdir?
None. You should use python's standard way of manipulating the current working directory.
Just note that it is not thread safe, so you might have problems if executing tasks in parallel.
If you explicitly use CmdAction
(not just pass a string) you could use the subprocess.Popen
's cwd
parameter.