Giters

akka / alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Home Page:https://doc.akka.io/docs/alpakka/current/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Alpakka - BQ behaviour

guda249 opened this issue · comments

commented

This is w.r.t to Big Query alpakka library I have following questions.

  1. There are no exception logged or any information is logged , when the BQ servers are not reachable, neither the Application shuts down.
  2. When the Google Credentials json are not passed, the application just logs the info and remains in the running state.

BigQuery.insertAll API accepts case classes for marshalling.

  1. Given the limitation of case class fields to 22. how to pass any data which has higher number of columns > 22 .
  2. Currently there is no provision to set the values for skipInvalidRows and ignoreUnknownValues, the values are set to None by default.

Suggestion needed on the following,

  1. Is it the expected behaviour w.r.t point 1 & 2 ? The ideal behaviour would be the application should shut down, as the upstream servers are not reachable, or retry the communication in exponential manner (in case if persistent connection are maintained, which is not the current case). In the worst case, there should be some information logged at error/info level. Or am i missing any configs which can give the expected behaviour.
  2. Any workaround for the Point 3.
  3. Are there are config's which can be used to pass these values mentioned in Point 4.

Version : 3.0.4