smarthi / recipes

The Immerok Apache Flink Cookbook is a collection of examples of Apache Flink applications in the format of "recipes". Each recipe explains how you can solve a specific problem by leveraging one or more of the APIs of Apache Flink. The recipes can be extended or provide a basis for solving your requirements with Apache Flink.

Recipes

Each recipe is a self-contained module.

Recipe overview

1. Deserializing JSON from Kafka -> Detailed explanation
2. Continuously Reading CSV Files -> Detailed explanation
3. Exactly-once with Apache Kafka® -> Detailed explanation
4. Writing an Application in Kotlin™ -> Detailed explanation
5. Joining and Deduplicating Data -> Detailed explanation
6. Splitting Apache Kafka® events -> Detailed explanation
7. Capturing Late Data -> Detailed explanation
8. Creating Dead Letter Queues -> Detailed explanation
9. Using Session Windows -> Detailed explanation
10. Serializing Timestamped Data -> Detailed explanation
11. Reading Apache Kafka Headers -> Detailed explanation
12. Upgrading Flink (Table API) -> Detailed explanation
13. Alerting when Problems Persist -> Detailed explanation
14. Migrating state away from Kryo -> Detailed explanation

Requirements

• Maven
• Java 11

IDE setup

Spotless

These instructions assume you use IntelliJ IDEA.

• Install the latest version of the google-java-format plugin
  • Note: For the time being the custom version used in Flink development is also fine.
• Enable the google-java-format as explained here
• Settings/Preferences -> Editor -> Code style
  • Tick "Enabled Editorconfig support"
  • Scala/Kotlin -> Imports -> Clear the import layout at the bottom
    • This is only relevant for the Scala/Kotlin recipes; you can likely skip this and just call spotless from the command-line
• Settings/Preferences -> Tools -> Actions on Save
  • Enable "Reformat code" and select Java/Scala/Kotlin file types
  • Enable "Optimize imports"

Note: You can also run mvn spotless:apply on the command-line / IDE to reformat the code.