snowplow-incubator / snowplow-google-cloud-storage-loader

Dataflow job to dump the content coming from a PubSub subscription into Cloud storage

Home Page:https://snowplowanalytics.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Change base docker image to k8s-dataflow:0.1.0

peel opened this issue · comments

k8s-dataflow:0.1.0 adds a backoff mechanism for deploying dataflow jobs.
Without backoff all the jobs have been failing upon deploy because of missing async-created terraform resources.
The mechanism allows for images to allow for retrying startup for given period of time.