arelangi / ProduceData

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Produce Data

This repo will generate fake data for the SurveyFunky mock app.

  • GoFakeIt library will be used to generate the data

We will start working on the User object

  • Generate data to DB
  • Publish data to Kafka
    • Register schema in schema registry
    • Write to Kafka
  • Ensure data lands in Hudi
  • Ensure data is queryable by ad-hoc analysis tools

02/02/2022

  • Refactor User.go and abstract away the push to Kafka
  • Setup a proper partition path to be able to query data (Test non-date partitions)

About


Languages

Language:Go 100.0%