guangxush / Mountain

Data Migration By Kafka

SpringBoot_Kafka

Using SpringBoot send data to Kafka

Function

1. producer send data to kafka
2. customer receive data from kafka
3. post producer send data by web services
4. file producer send data by file in the servers
5. monitor the exception and data process by email
6. using ssh wrapper connect the server
7. listen the file on the server every day by scheduled task
8. port forward connection via ssh tunnel
9. using the private key connect the server without password
10. get LineIterators to lazy load data and avoid load all data into memory at once

Technology Stack

• SpringBoot(web,mail,ScheduleTask,AOP,jcraft,json)
• Kafka
• Linux shell