Is it bad to capture a bare `Exception`?

Question

Is it bad to capture a bare `Exception`?

MaxG87 opened this issue 3 years ago · comments

I learned that catching Exception is a dangerous shortcut. It does not communicate what is expected to fail, it might silence arbitrary unexpected errors (e.g. AttributeError where ValueError is expected) and finally it also catches severe misconditions like MemoryError. Thus usually I try to be as specific as possible.

What is your position on catching Exception? Is it something thats acceptable in Tryceratops?

Guilherme Latrova · Answer 1 · Thu Jul 22 2021 06:34:25 GMT+0800 (China Standard Time)

First of all, that's a very good question. I'm about to share my personal opinion (which is not law or final word), and I'd expect you to criticize it.

Defining an exception

I see an exception as "something failed", and not "something failed BECAUSE X". To make it more obvious, let's have some examples.

This is our function that may raise KeyError:

class Storage:
    def save_key_to_database(self, key: str):
        value = self._internal_dict[key]  # may raise KeyError
        self.save(value)

As you can see, I'm not raising any exception.

Usage:

def long_process():
    try:
        step_one()
        step_two()
        storage.save_key_to_database("invalid_key") # it's not present in `_internal_dict`
        one_more_function()
   except KeyError:  # Let's say you want to be careful with this, so you're capturing it just in case.
       # What are you going to do here? How does this except block helps you fixing the issue?
       return "Something that I don't know failed, try again later"
   else:
       return "Hey customer, everything worked fine"

This is the situation that IMHO doesn't help at all. You run several steps/functions/methods/whatever and you captured the KeyError exception.

The issues: You don't know where it came from

First, you don't know for sure which function raised it, maybe it was step_one! Or maybe it was step_two. Who knows, maybe one_more_function raised it.

The issues: What the hell can you do???

Let's say that python (by some dark magic) now tells you the caller who caused the issue, what do you do in the case of KeyError, AttributeError, ValueError?

I would just log as error or maybe critical and trigger some alarm to have a developer investigating / fixing it as soon as possible.

You should be able to react, you don't lose any context!

The ideal solution in my opinion is:

class Storage:
    def save_key_to_database(self, key: str):
        try:
             value = self._internal_dict[key]  # may raise KeyError
             self.save(value)
        except Exception as e:
             raise CouldntSaveToDatabase(key) from e

Please note the from e usage, you're not losing the cause. You aren't hiding the issue, you're just encapsulating it.

Let's see what you can do now:

def long_process():
    try:
        step_one()
        step_two()
        storage.save_key_to_database("invalid_key") # it's not present in `_internal_dict`
        one_more_function()
   except CouldntSaveToDatabase as e:
        logger.exception(f"Couldn't save to database using key '{e.key}'")
        undo_step_one()
        return "Sorry, we're having temporary issues. We're working hard to solve that, try again later!"
    else:
       return "Hey customer, everything worked fine"

First, let's note why it's better:

Now you know exactly WHAT it (not why) failed
You can react to it!
You log the FULL exception, with the CAUSE (Key Error), you DO NOT SILENCE anything (See example)
You know what "excuse" to tell your customer. I bet you don't want to tell the final user that you had a KeyError problem, do you? It would be worse for security (consider a database connection failed, and then the database returns something like 'user root is not allowed from address XXX.XXX.XXX.XXX'), and also as UX (what the user should do if he receives a KeyError?)

Regarding "2. You can react to it:" it allows you to, for example, invoke undo_step_one() inside the except block. Most of the time the cause of the issue should be investigated, you just need to identify WHAT failed.

If you still didn't buy it, I can share with you a real-life example from a system running in production. I wrote about this in a blog post recently, feel free to read if you want

Conclusion

I learned that catching Exception is a dangerous shortcut.

If you don't log, store it somewhere, don't keep the context, don't trigger alarms, then yes. Otherwise, I don't think so.

It does not communicate what is expected to fail, it might silence arbitrary unexpected errors (e.g. AttributeError where ValueError is expected) and finally it also catches severe misconditions like MemoryError.

If you do it correctly (logging or etc), you don't silence anything

Thus usually I try to be as specific as possible.

Be specific, but prefer precision (what) over accuracy (how/why).

Is it something thats acceptable in Tryceratops?

IMHO yes

Final reminder: This is my personal opinion. What do you think about it @MaxG87 ? Do you see any possible flaws in my logic? I'm glad to hear if you do.

Guilherme Latrova · Answer 2 · Mon Jul 26 2021 19:41:08 GMT+0800 (China Standard Time)

Since I didn't hear back from you @MaxG87 I'll be closing this. Happy to reopen if you disagree/ have better ideas.

Max Görner · Answer 3 · Tue Jul 27 2021 16:22:24 GMT+0800 (China Standard Time)

Sorry, I was offline over the weekend. I really appreciate your thorough argumentation. It was very insightful for me and I believe it will be very useful for others driven here by search engines. However, I fear I disagree in your main conclusions. Also, you managed to dodge one of my main arguments.

My Guiding Principles About Exceptions

My position about exceptions is that exceptions are a special kind of control flow. They should be used for one of two reasons: (a) to restore consistency of the state of the program or (b) to improve the error messages, e.g. write a dedicated log message and reraise the exception.

Neither (a) nor (b) can be done for unanticipated errors. Since they were not anticipated, one does not know what can be done to restore consistency. Similar, since they were not anticipated, we cannot tell anything specific about them that is not contained in the stacktrace already. Thus, I am convinced exceptions are not are not meant to handle unanticipated errors.

Since exceptions are control flow constructs for anticipated circumstances, the same rules that apply to other control flow apply here too. In particular, try-except-blocks should be as narrow as possible.

Also, I think, that exceptions are the least desirable of all control flow constructs. They are very handy here and there, they help to escalate error conditions without complicating every function in the call stack, but if other
means exist I like to use these.

Applying the Principles to Your Example

So lets apply these principles to your example. I will repeat it here for convenience:

class Storage:
    def save_key_to_database(self, key: str):
        value = self._internal_dict[key]  # may raise KeyError
        self.save(value)


def long_process():
    try:
        step_one()
        step_two()
        storage.save_key_to_database("invalid_key") # it's not present in `_internal_dict`
        one_more_function()
    except KeyError:  # Let's say you want to be careful with this, so you're capturing it just in case.
       # What are you going to do here? How does this except block helps you fixing the issue?
       return "Something that I don't know failed, try again later"
    else:
       return "Hey customer, everything worked fine"

I agree with you when you argue that its hard to know which step raised the KeyError. But this is caused by the overly wide exception block. So lets narrow it down:

def long_process():
    step_one()
    step_two()
    key = "invalid_key"
    try:
        storage.save_key_to_database(key)
    except KeyError:
        # Now we now the cause for the exception and can handle it properly.
        denylist.add(key)
        logger.error(f"Key {key} is unknown to storage {storage}. Added to denylist.")
        return
    one_more_function()
    return "Hey customer, everything worked fine"

In my example we exactly know which statement caused the exception. This allows
us to handle the situation properly. To go through my list of principles, we have:

a control flow block as narrow as possible
a repaired program state
a very specific and helpful error message

One might object now that any KeyError raised by step_one, step_two or one_more_function will cause the program to crash. I argue that this is a feature, not a problem. Because we do not anticipate KeyError from these steps, we cannot handle them at all. Neither do we know how what to say about errors in these steps. Letting the program crash will not expose us to the stack trace and ensure that we learn about the inconsistent state as soon as possible.

Also, your fix of the example, you introduced CouldntSaveToDatabase, lets the program crash in case of KeyError too. I am aware that you were improving your example later on. Indeed, I also think your evolutions of the code snippet are improvements. However, I think they improved the wrong problem. I think by narrowing down the try-except-block I achieved the same effect too but with less complexity.

Catching `Exception` Means Catching Things That Cannot be Handled

One argument was completely uncovered by your reply. It is the argument that catching Exception means catching MemoryError, BrokenPipe, KeyboardInterrupt, ModuleNotFound and the like.

Every except Exception will catch the above mentioned exceptions and possibly some other, similar fatal. The exception MemoryError is so severe that there is no guarantee one can recover from it at all. All the others likely should not be treated if they were not expected. Of course, if one needs to tidy up before terminating the program upon Control-C, one may catch it. Of course, if one writes a Shell utility one does want to silence a BrokenPipe.

But if one does requests.get(url) or storage.save_key_to_database(key) these very severe exceptions, imho, should be left alone and not be caught and handled.

Conclusion

After working with your reply carefully for some hours now, I don't really see how we disagree. The issue was about catching Exception and your fix to your problem was to replace a generic exception with a more concrete one. This exactly what I am arguing for.

I don't agree with your conclusion that properly handling exceptions is enough. I think in most circumstances MemoryError, ModuleNotFound etc. cannot be handled. This applies especially when one did not anticipate them and caught
Exception.

Since we are talking about a linter, I think that relying on users to do logging, triggering alarms and keeping context is inappropriate. If users were able to work thoroughly they would not need a linter. Since working thoroughly does not scale one has to except careless mistakes. A linter should not accept circumstances that are acceptable only in combination with thorough handling.

You prefer to know what failed over why it failed. However, you achieve that by catching a dedicated exception, which I would count as a case against except Exception.

Final Conclusion

Is it something that is acceptable in Tryceratops?
IMHO yes

I still consider that to be too dangerous.

But it is your tool and your codebase, so my judgement must not be your judgement.

I already found some of the generated warnings helpful. I think Tryceratops will not be less useful if it will not warn about a condition I don't like.

Guilherme Latrova · Answer 4 · Tue Jul 27 2021 18:05:42 GMT+0800 (China Standard Time)

Amazing feedback!

My position about exceptions [...] They should be used for one of two reasons: (a) to restore consistency [...] or (b) to improve the error messages [...]
Neither (a) nor (b) can be done for unanticipated errors. Since they were not anticipated, one does not know what can be done to restore consistency.

I partially agree.

For (a) There are certain operations (if I know they failed) that I can use to (or try to) restore consistency.
e.g. If I failed to save a resource on the database (not sure why), so I need to delete the resource from an API

For (b) I think is always possible!

try:
   ...
except CouldntSaveToDatabase: 
    logger.exception("failed to save on database")  # <-- I improved the log message!

Thus, I am convinced exceptions are not are not meant to handle unanticipated errors.

Ok, I believe you're right on this one

Also, I think, that exceptions are the least desirable of all control flow constructs. They are very handy here and there, they help to escalate error conditions without complicating every function in the call stack, but if other
means exist I like to use these.

I agree. I think of exceptions as interruptions to regular flows (ironically, an exception to the flow haha).

In my example we exactly know which statement caused the exception. This allows
us to handle the situation properly. To go through my list of principles, we have:

You're correct. You solved it. Now it's a matter of "personal design taste". I want to avoid too many try blocks because it's hard to read.

But again: you're right. You solved it.

Catching Exception Means Catching Things That Cannot be Handled

One argument was completely uncovered by your reply.

Indeed, the truth is that I never had these issues before, but let me try to give you a better answer on this.

Every except Exception will catch the above mentioned exceptions and possibly some other, similar fatal.

KeyboardInterrupt probably not (which is good):

>>> isinstance(KeyError(), Exception)
True
>>> isinstance(KeyboardInterrupt(), Exception)
False

You're correct regarding MemoryError, BrokenPipe, and maybe others. I never faced them ever in my entire life. Have you before?

Anyway, I don't think it harms your software because IMHO you can't do anything at all.

try:
   ...
except MemoryError:
   # What can I do besides logging?
except BrokenPipe:
  # What can I do besides logging?

I want my (thus it's a personal opinion) software to be able to react to problems, not necessarily to fix them. So:

try:
   ...
except ThisFailed:
   # What should I do when "This" fails?
   logger.exception("this failed!")
   revert_this()
except Exception:
   # This is unknown and unexpected! Just log (including MemoryError, etc)

It's up to you! IMHO if you know what to do when facing a MemoryError, go for it! Capture it! Handle it! Since I don't know, and I assume other people in my team wouldn't know as well I prefer to:

Capture WHAT failed
Log the WHY (from the stack trace, not silencing anything)
Notify the engineering team to read the full log and fix it

I think that relying on users to do logging, triggering alarms and keeping context is inappropriate. If users were able to work thoroughly they would not need a linter.

I agree! That's why Tryceratops enforces some rules, and that's why it's on the roadmap to include, for example, logger.exception over logger.error, and some others people aren't aware of. Maybe I can't make it perfect, but I'll do my best to give the developer the tools and power to define what to ignore, what to follow. So it's up to you! 💪

You prefer to know what failed over why it failed. However, you achieve that by catching a dedicated exception, which I would count as a case against except Exception.

Indeed, we agree on the overall, we disagree on the details, which is fine. Sometimes I personally like to wrap the global Exception to make it more granular regarding the "WHAT".

But it is your tool and your codebase, so my judgement must not be your judgement.

The beauty of open source is that, actually, it's our tool and our codebase. You can always create your own fork and adjust rules to match your needs, and I bet there would be a community supporting your work (the same way you're supporting Tryceratops so far! 🥰)

Disclaimer to others: This discussion IMHO will never end, because there's no "RIGHT" so I welcome you to criticize both me and @MaxG87 , maybe you disagree on both of us - NEAT! Share your thoughts.

Consider that even Guido dislikes Black formatting, and that's fine. Keeping the discussion is worthy! 🔥