26.3 Exception Design Tips

By and large, exceptions are easy to use in Python. The real art behind them is deciding how specific or general your except clauses should be, and how much code to wrap up in try statements. Let's address the second of these first.

26.3.1 What Should Be Wrapped

In principle, you could wrap every statement in your script in its own try, but that would just be silly (the try statements would then need to be wrapped in try statements!). This is really a design issue that goes beyond the language itself, and becomes more apparent with use. But here are a few rules of thumb:

• Operations that commonly fail are generally wrapped in try statements. For example, things that interface with system state, such as file opens, socket calls, and the like, are prime candidates for try.

• However, there are exceptions to the prior rule—in simple scripts, you may want failures of such operations to kill your program, instead of being caught and ignored. This is especially true if the failure is a show-stopper. Failure in Python means a useful error message (not a hard crash), and this is often the best outcome you could hope for.

• Implement termination actions in try/finally statements, in order to guarantee their execution. This statement form allows you to run code whether exceptions happen or not.

• It is sometimes more convenient to wrap the call to a large function in a single try statement, rather than littering the function itself with many try statements. That way, all exceptions in the function percolate up to the try around the call, and you reduce the amount of code within the function.

26.3.2 Catching Too Much

On to the issue of handler generality. Because Python lets you pick and choose which exceptions to catch, you sometimes have to be careful to not be too inclusive. For example, you've seen that an empty except clause catches every exception that might be raised while the code in the try block runs.

That's easy to code and sometimes desirable, but you may also wind up intercepting an error that's expected by a try handler higher up in the exception nesting structure. For example, an exception handler such as the following catches and stops every exception that reaches it—whether or not another handler is waiting for it:

def func(  ):
    try:
        ...                # IndexError is raised in here.
    except:
        ...                # But everything comes here and dies!

try:
    func(  )
except IndexError:         # Needed here
    ...

Perhaps worse, such code might also catch system exceptions. Even things like memory errors, programming mistakes, iteration stops, and system exits raise exceptions in Python. Such exceptions should not usually be intercepted.

For example, scripts normally exit when control falls off the end of the top-level file. However, Python also provides a built-in sys.exit call to allow early terminations. This actually works by raising a built-in SystemExit exception to end the program, so that try/finally handlers run on the way out, and special types of programs can intercept the event.^[2] Because of this, a try with an empty except might unknowingly prevent a crucial exit, as in file exiter.py:

^[2] A related call, os._exit also ends a program, but is an immediate termination—it skips cleanup actions and cannot be intercepted with try/except or try/finally. It is usually only used in spawned child processes—a topic beyond this book's scope. See the library manual or Programming Python, Second Edition (O'Reilly) for details.

import sys

def bye(  ):
    sys.exit(40)             # Crucial error: abort now!

try:
    bye(  )
except:
    print 'got it'           # Oops--we ignored the exit
print 'continuing...'


% python exiter.py
got it
continuing...

You simply might not expect all the kinds of exceptions that could occur during an operation. In fact, an empty except will also catch genuine programming errors, which should also be allowed to pass most of the time:

mydictionary = {...}
...
try:
    x = myditctionary['spam']     # Oops: misspelled 
except:
    x = None                      # Assume we got KeyError.
...continue here...

The coder here assumes the only sort of error that can happen when indexing a dictionary is a key error. But because the name myditctionary is misspelled (it should say mydictionary), Python raises a NameError instead for the undefined name reference, which will be silently caught and ignored by the handler. The event will incorrectly fill in a default for the dictionary access, masking the program error. If this happens in code that is far removed from the place where the fetched values are used, it might make for a very interesting debugging task.

As a rule of thumb, be specific in your handlers—empty except clauses are handy, but potentially error-prone. In the last example, for instance, you should usually say except KeyError: to make your intentions explicit, and avoid intercepting unrelated events. In simpler scripts, the potential for problems might not be significant enough to outweigh the convenience of a catch-all. But in general, general handlers are generally trouble.

26.3.3 Catching Too Little

Conversely, handlers also shouldn't be too specific. When listing specific exceptions in a try, you catch only what you actually list. This isn't necessarily a bad thing either, but if a system evolves to raise other exceptions in the future, you may need to go back and add them to exception lists elsewhere in the code.

For instance, the following handler is written to treat myerror1 and myerror2 as normal cases, and treat everything else as an error. If a myerror3 is added in the future, it is processed as an error unless you update the exception list:

try:
    ...
except (myerror1, myerror2):    # What if I add a myerror3?
    ...                         # Nonerrors
else:
    ...                         # Assumed to be an error

Careful use of class-based exceptions can make this trap go away completely. As we learned in the prior chapter, if you catch a general superclass, you can add and raise more specific subclasses in the future without having to extend except clause lists manually:

try:
    ...
except SuccessCategoryName:     # What if I add a myerror3?
    ...                         # Nonerrors
else:
    ...                         # Assumed to be an error

Whether you use classes here or not, a little design goes a long way. The moral of the story is that you have to be careful not to be too general or too specific in exception handlers, and have to pick the granularity of your try statement wrapping wisely. Especially in larger systems, exception policies should be a part of the overall design.