commsor / titanoboa

Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.

Home Page:https://titanoboa.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add ability for Workers to self-initate restart in case of a fatal error.

mikub opened this issue · comments

Normally when a worker encounters an error that can't be handled (e.g. an error is fired again from inside of the catch clause) the processing loop as well as worker thread will terminate.

Good example of this type of error is - in distributed setup - if the underlying MQ connection is terminated then basically all the workers stop and all workflow processing grinds to a halt. Currently such situation has to be monitored/discovered/handled by admins.

Proposed solution:
Add a property :restart-workers-on-error to systems so as it can be configurable to initiate a restart in case of such fatal error.