kafka2http
Simple Example
You just have some topic and you want to hit some API endpoint when you get a message.
- Get a message from the users topic
- POST to /users/create
kafka:
topic: ping
group: a
http:
method: POST
url: http://myapi.com/users/pong
Templated requests based on body
You can template a http request based on the contents of the body.
http:
method: POST
url: http://myapi.com/users/create
body: |
{
userId: ${msgBody.user.id},
lovesCats: True
}
More sophisticated example
The above examples can be useful but often there are other concerns.
- What if you want to keep retrying?
- What if we want to only retry in speific cases?
- What about authentication?
The following example demonstrates:
- Get a message from the users topic
- POST to /users/create
- Templatize body of request based on message contents
- Retry but only on 500 errors
- If either there is a failure or the retry limit is reached put it in another topic for retrivial later (dead letter queue)
- Throttle service so that it doesn't overwhelm the receiving service
kafka:
topic: users
group: a
http:
method: POST
url: http://myapi.com/users/create
body: |
{
userId: ${msgBody.user.id},
lovesCats: True
}
headers:
Content-Type: application/json
authorization: Bearer
retry:
onlyRetryOn:
- statusCode: 5xx
onlyRetryOn:
5**
deadLetterQueue: users-dlq
Limitations
- Only supports json encoded messages for now.
- This is meant to be a simple service and you might have more sophisticated needs.
- If we are not able to accommodate you may want to build your own service or make a fork.