How do i handle errors?
Plaoo opened this issue · comments
After implementing logpipe on Django, everything worked perfectly, but it happened that a message arrived with all fields empty, this caused Kafka to freeze, which remained on that message and did not let the others pass. I have implemented error handling like this, but I would like to know how I could handle the error by sending messages containing errors to another queue.
if type == 'add':
if topic:
try:
topic = f"{topic}_posts"
producer = Producer(topic, serializers.PostCreateKafkaSerializer)
producer.send(data)
logger.info("Try to Send to Kafka")
except Exception as e:
resp = requests.post(url, headers = {"Content-Type": "application/json", "Authorization": f"Token {token}"}, data=json.dumps(data))
print(e)
try:
resp.json()
logger.warning("Fallback send with POST, Kafka Problems!")
except Exception:
traceback.print_exc()
raise NotExpectedException("Error Post")
else:
try:
resp = requests.post(url, headers = {"Content-Type": "application/json", "Authorization": f"Token {token}"}, data=json.dumps(data))
resp.json()
logger.warning("No topic send by Post")
except Exception as error:
traceback.print_exc()
logger.error("Error Post")
raise NotExpectedException("Error Post")
Basically I do a fallback with a post in case I am unable to send it via Kafka, connection problems or Kafka down, what I would like to avoid is Kafka blocking when I send a message that is not in order. Not if it is possible to manage it directly from logpipe.