naveenkumaran / serverless-kotlin-kafka

Serverless Kotlin Kafka

ws-to-kafka

Dev (TestContainer Kafka):

./gradlew :ws-to-kafka:bootRun

consume messaged with schema

docker exec -it schema-registry /usr/bin/kafka-json-schema-console-consumer --topic mytopic --bootstrap-server broker:9092

Prod (External Kafka):

export KAFKA_BOOTSTRAP_SERVERS=YOUR_BOOTSTRAP_SERVERS
export KAFKA_USERNAME=YOUR_CCLOUD_KEY
export KAFKA_PASSWORD=YOUR_CCLOUD_PASSWORD
export SCHEMA_REGISTRY_URL=YOUR_CCLOUD_SCHEMA_REGISTRY_URL
export SCHEMA_REGISTRY_KEY=YOUR_SCHEMA_REGISTRY_USERNAME
export SCHEMA_REGISTRY_PASSWORD=YOUR_SCHEMA_REGISTRY_PASSWORD

./gradlew :ws-to-kafka:run

containerize & run

./gradlew :ws-to-kafka:bootBuildImage --imageName=skk-ws-to-kafka

# using env vars from above
docker run -it \
  -eKAFKA_BOOTSTRAP_SERVERS=$KAFKA_BOOTSTRAP_SERVERS \
  -eKAFKA_USERNAME=$KAFKA_USERNAME \
  -eKAFKA_PASSWORD=$KAFKA_PASSWORD \
  -eSCHEMA_REGISTRY_URL=$SCHEMA_REGISTRY_URL \
  -eSCHEMA_REGISTRY_KEY=$SCHEMA_REGISTRY_KEY \
  -eSCHEMA_REGISTRY_PASSWORD=$SCHEMA_REGISTRY_PASSWORD \
  skk-ws-to-kafka

web-ui

Dev (TestContainer Kafka):

./gradlew :web-ui:bootRun

View the Web UI: http://localhost:8080

---

TODO

Prod (External Kafka):

export KAFKA_BOOTSTRAP_SERVERS=YOUR_BOOTSTRAP_SERVERS
export KAFKA_USERNAME=YOUR_CCLOUD_KEY
export KAFKA_PASSWORD=YOUR_CCLOUD_PASSWORD
export KSQLDB_ENDPOINT=YOUR_KSQLDB_ENDPOINT
export KSQLDB_USERNAME=YOUR_KSQLDB_USERNAME
export KSQLDB_PASSWORD=YOUR_KSQLDB_PASSWORD

./gradlew :web-ui:run

containerize & run

./gradlew :web-ui:bootBuildImage --imageName=skk-web-ui

# using env vars from above
docker run -it \
  -eKAFKA_BOOTSTRAP_SERVERS=$KAFKA_BOOTSTRAP_SERVERS \
  -eKAFKA_USERNAME=$KAFKA_USERNAME \
  -eKAFKA_PASSWORD=$KAFKA_PASSWORD \
  -eKSQLDB_ENDPOINT=$KSQLDB_ENDPOINT \
  -eKSQLDB_USERNAME=$KSQLDB_USERNAME \
  -eKSQLDB_PASSWORD=$KSQLDB_PASSWORD \
  skk-web-ui

todo

• schema stuff

  • json schema used

• parse tags

• ksqldb stateful (total favorites)

• ksql transform (language groupings)

• web event consumer (websocket stream + pull query)