algorand / conduit

Algorand's data pipeline framework.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Rework pipeline process management

winder opened this issue · comments

Subject of the issue

The pipeline object is managed using a context with a cancel function and a wait group. This setup is strange.

Rather than managing the process internal to the pipeline, Start() should be renamed to Run(), and Run() should be blocking. If the context is cancelled Run() can call Stop() automatically.

The caller would then use a waitgroup and run the pipeline in a go-routine if it wants to. Run() could also return the error directly rather than implementing setError and Error.

Closing in favor of #100