Giters

akka / alpakka-kafka

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Home Page:https://doc.akka.io/docs/alpakka-kafka/current/home.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Implement serde flow using AWS Glue schema registry

burakkose opened this issue · comments

commented

Description

AWS Glue provides a feature for managing schemas.

https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html

awslab implemented a client library for Kafka to use schema registry for serializing and deserializing avro recrods.

https://github.com/awslabs/aws-glue-schema-registry

The kafka flow should be able utilize it.

commented

Can it not be used as-is? What are the challenges?

commented

Hi Sean, you are right, this can be actually used as-is. I guess what I was thinking is to construct a flow that can stream Scala's case-class to Kafka using the library that can handle the conversion to Avro's records out of the box. However, that is also auto of the scope of Alpakka's Kafka library.

Please feel free to close the issue.