thelabnyc / django-logpipe

Automated mirror of https://gitlab.com/thelabnyc/django-logpipe

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How do i handle errors?

Plaoo opened this issue · comments

After implementing logpipe on Django, everything worked perfectly, but it happened that a message arrived with all fields empty, this caused Kafka to freeze, which remained on that message and did not let the others pass. I have implemented error handling like this, but I would like to know how I could handle the error by sending messages containing errors to another queue.

    if type == 'add':
        if topic:
            try:
                topic = f"{topic}_posts"
                producer = Producer(topic, serializers.PostCreateKafkaSerializer)
                producer.send(data)
                logger.info("Try to Send to Kafka")
            except Exception as e:
                resp = requests.post(url, headers = {"Content-Type": "application/json", "Authorization": f"Token {token}"}, data=json.dumps(data))
                print(e)
                try:
                    resp.json()
                    logger.warning("Fallback send with POST, Kafka Problems!")
                except Exception:
                    traceback.print_exc()
                    raise NotExpectedException("Error Post")
        else:
            try:
                resp = requests.post(url, headers = {"Content-Type": "application/json", "Authorization": f"Token {token}"}, data=json.dumps(data))
                resp.json()
                logger.warning("No topic send by Post")
            except Exception as error:
                traceback.print_exc()
                logger.error("Error Post")
                raise NotExpectedException("Error Post")

Basically I do a fallback with a post in case I am unable to send it via Kafka, connection problems or Kafka down, what I would like to avoid is Kafka blocking when I send a message that is not in order. Not if it is possible to manage it directly from logpipe.