Implement serde flow using AWS Glue schema registry
burakkose opened this issue · comments
Description
AWS Glue provides a feature for managing schemas.
https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html
awslab implemented a client library for Kafka to use schema registry for serializing and deserializing avro recrods.
https://github.com/awslabs/aws-glue-schema-registry
The kafka flow should be able utilize it.
Can it not be used as-is? What are the challenges?
Hi Sean, you are right, this can be actually used as-is. I guess what I was thinking is to construct a flow that can stream Scala's case-class to Kafka using the library that can handle the conversion to Avro's records out of the box. However, that is also auto of the scope of Alpakka's Kafka library.
Please feel free to close the issue.