confluentinc / kafka-connect-bigquery

A Kafka Connect BigQuery sink connector

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

schemaless ( JSON events ) to bq fail to create table

raphaelauv opened this issue · comments

I want archive the topic _schemas in bigquery ( it contains only events with a JSON key and a JSON value but not serialized with a JSON schema )

so I want use the schemaless functionnality of the connector but it fail on

com.wepay.kafka.connect.bigquery.exception.BigQueryConnectException: Failed to unionize schemas of records for the table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=schema_metadata, tableId=_schemas}} Caused by: Could not convert to BigQuery schema with a batch of tombstone records. at 

com.wepay.kafka.connect.bigquery.SchemaManager.getTableInfo(SchemaManager.java:297) at 

com.wepay.kafka.connect.bigquery.SchemaManager.createTable(SchemaManager.java:240) at 

com.wepay.kafka.connect.bigquery.write.row.AdaptiveBigQueryWriter.attemptTableCreate(AdaptiveBigQueryWriter.java:161) at

com.wepay.kafka.connect.bigquery.write.row.AdaptiveBigQueryWriter.performWriteRequest(AdaptiveBigQueryWriter.java:102) at

com.wepay.kafka.connect.bigquery.write.row.BigQueryWriter.writeRows(BigQueryWriter.java:112) at 

did someone found a way to send kafka events without schema to bq ?

thanks