jmleoni / batch-demo

A demo illustrating the use of Spring batch for use with AWS batch

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AWS Batch and spring Batch demo

This small project is a simple Spring batch application that reads a CSV file from AWS S3 and parses it using Open CSV multithreaded parser. It may run on AWS Batch once package as a docker container (see Dockerfile)

With 40 cores and enough RAM it parses 29Millions rows (10Gb of data) in less than 5 minutes.

About

A demo illustrating the use of Spring batch for use with AWS batch


Languages

Language:Java 97.3%Language:Dockerfile 2.7%