Initially inspired by this medium article ... (was deleted meanwhile, maybe because of the lack of quality)
I tried to improve my go skills and added a few features:
- better error handling
- better concurrency control for quitting the workers
- better sql performance by using batches
Still the code is not very clean, but it works.
Next thing is to provide better documentation or a small blog article.
There is a docker compose file included in the repo to provide a mariadb instance for testing. It comes together with adminer a minimalistic web admin ui for mysql.
Test data can be downloaded with
wget http://downloads.majestic.com/majestic_million.csv