runprism / prism

Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.

Home Page:https://runprism.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`done` method in PrismTask class

prism-admin opened this issue · comments

Is your feature request related to a problem? Please describe.

When running a Prism project, I want to be able to skip tasks that are already "Done". I can do this while running individual tasks (i.e., if I run a specific Prism task, then the task will skip upstream tasks that have a target). However, I want to be able to do this when running the project as a whole.

Describe the solution you'd like

  • Add a done method to the PrismTask class. This function must return either True of `False.
class PrismTask:
    ...
    def done(self, hooks):
        # do something here
        return True
  • Users can overwrite this this function in their own task definitions.
  • When running each task in a project, call done() before calling exec()

Describe alternatives you've considered
N/A

Additional context
N/A

So the done method is meant to be a protected method called by execute. That can be over written by sub classes. However the done method will be only accessible by the execute method correct? No one else is meant to call this method?

So the done method is meant to be a protected method called by execute. That can be over written by sub classes. However the done method will be only accessible by the execute method correct? No one else is meant to call this method?

Yes, that's correct. Users will be able to define custom logic for their task's done method, but they will never need to call it directly.