marsprobe / camus-sweeper-cdh

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

#What?

Adopted the Camus Sweeper Project (code) to tackle the Small File Issue together with Camus (code)

Camus Sweeper aggregates the hourly Avro files to daily Avro files

Camus Sweeper needs Avro-Schema files. Since I don't have a Schema Registry I added a configuration parameter for that:

# define the topic to schema class mappings here
# f.e. for topic eu_fakod_app_search_term you should configure:
eu_fakod_app_search_term.camus.sweeper.avro.schema.class=eu.fakod.app.search.term.SchemaClass

#Little more Information

About


Languages

Language:Java 100.0%