koron / csv2avro

koron/csv2avro

Convert CSV/TSV file to AVRO file.

How to install

$ go install github.com/koron/csv2avro@latest

Features

• Detect CSV or TSV by extension of file name
• Auto detect a header row

Usage

$ csv2avro -schema {SCHEMA} [-input {INPUT}] [-output {OUTPUT}] [OPTIONS]

• SCHEMA: AVRO schema, mandatory
• INPUT: optional, default is STDIN It will be treated as TSV when a file name ends with ".tsv". Otherwise it is treated as CSV.
• OUTPUT: optional, default is STDOUT

Other options:

• -tsv Force input as TSV.

Example:

$ csv2avro -schema my_shcema.avsc < input.csv > output.avro

$ csv2avro -schema my_shcema.avsc -tsv < input.tsv > output.avro

Supported Avro's types

• string
• int
• long
• float
• double
• boolean
• null - always null.

Misc for development

Test command:

$ go build
$ ./csv2avro -schema testdata/sample1.avsc -input testdata/sample1.csv -output testdata/sample1.avro