add json support
chainray01 opened this issue · comments
CSV is not very convenient for subsequent development. Can you please add json optional support?
Hi chainray01,
flume-ng-sql-source is a custom component of Apache Flume. It takes rows from supported databases and creates Flume Events. A flume event is made of a header and a body. End of source processing.
Whatever you need from this point forward, you need to add a component called Sink. If your Sink needs reading data as Json you may use another component called Interceptor, for intercepting events and turning them into Json structured data.
Standard CSV is one of the most convenient way for subsequente development with structured SQL data, but Flume-sql-source is not dependent on csv, so we dont understand you. I think you mean to File_roll Fume Sink. It lets sink data to text plain file. It is really usefull for debugging.
My recommendation:
- Flume user guide
- try out Flume-enrichment-interceptor, it turns flume events into enriched flume events as Json structured data.
Please repoen if you think you can contribute in any way.
best