schemaless ( JSON events ) to bq fail to create table
raphaelauv opened this issue · comments
raphaelauv commented
I want archive the topic _schemas
in bigquery ( it contains only events with a JSON key and a JSON value but not serialized with a JSON schema )
so I want use the schemaless functionnality of the connector but it fail on
com.wepay.kafka.connect.bigquery.exception.BigQueryConnectException: Failed to unionize schemas of records for the table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=schema_metadata, tableId=_schemas}} Caused by: Could not convert to BigQuery schema with a batch of tombstone records. at
com.wepay.kafka.connect.bigquery.SchemaManager.getTableInfo(SchemaManager.java:297) at
com.wepay.kafka.connect.bigquery.SchemaManager.createTable(SchemaManager.java:240) at
com.wepay.kafka.connect.bigquery.write.row.AdaptiveBigQueryWriter.attemptTableCreate(AdaptiveBigQueryWriter.java:161) at
com.wepay.kafka.connect.bigquery.write.row.AdaptiveBigQueryWriter.performWriteRequest(AdaptiveBigQueryWriter.java:102) at
com.wepay.kafka.connect.bigquery.write.row.BigQueryWriter.writeRows(BigQueryWriter.java:112) at
did someone found a way to send kafka events without schema to bq ?
thanks