23.4 Class Gotchas

Most class issues can usually be boiled down to namespace issues (which makes sense, given that classes are just namespaces with a few extra tricks). Some of the topics in this section are more like case studies of advanced class usage than problems, and one or two of these have been eased by recent Python releases.

23.4.1 Changing Class Attributes Can Have Side Effects

Theoretically speaking, classes (and class instances) are all mutable objects. Just as with built-in lists and dictionaries, they can be changed in place, by assigning to their attributes. And like lists and dictionaries, this also means that changing a class or instance object may impact multiple references to it.

That's usually what we want (and is how objects change their state in general), but this becomes especially critical to know when changing class attributes. Because all instances generated from a class share the class's namespace, any changes at the class level are reflected in all instances, unless they have their own versions of changed class attributes.

Since classes, modules, and instances are all just objects with attribute namespaces, you can normally change their attributes at runtime by assignments. Consider the following class; inside the class body, the assignment to name a generates an attribute X.a, which lives in the class object at runtime and will be inherited by all of X's instances:

>>> class X:
...     a = 1        # Class attribute
...
>>> I = X(  )
>>> I.a              # Inherited by instance
1
>>> X.a
1

So far so good—this is the normal case. But notice what happens when we change the class attribute dynamically outside the class statement: it also changes the attribute in every object that inherits from the class. Moreover, new instances created from the class during this session or program get the dynamically set value, regardless of what the class's source code says:

>>> X.a = 2          # May change more than X
>>> I.a              # I changes too.
2
>>> J = X(  )             # J inherits from X's runtime values
>>> J.a              # (but assigning to J.a changes a in J, not X or I).
2

Is this a useful feature or a dangerous trap? You be the judge, but you can actually get work done by changing class attributes, without ever making a single instance. This technique can simulate "records" or "structs" in other languages. As a refresher on this technique, consider the following unusual but legal Python program:

class X: pass                          # Make a few attribute namespaces.
class Y: pass

X.a = 1                                # Use class attributes as variables.
X.b = 2                                # No instances anywhere to be found
X.c = 3
Y.a = X.a + X.b + X.c

for X.i in range(Y.a): print X.i       # Prints 0..5

Here, classes X and Y work like "file-less" modules—namespaces for storing variables we don't want to clash. This is a perfectly legal Python programming trick, but is less appropriate when applied to classes written by others; you can't always be sure that class attributes you change aren't critical to the class's internal behavior. If you're out to simulate a C "struct," you may be better off changing instances than classes, since only one object is affected:

class Record: pass
X = Record(  )
X.name = 'bob'
X.job  = 'Pizza maker'

23.4.2 Multiple Inheritance: Order Matters

This may be obvious, but is worth underscoring: if you use multiple inheritance, the order in which superclasses are listed in a class statement header can be critical. Python always searches your superclasses left to right, according to the order in the class header line.

For instance, in the multiple inheritance example we saw in Chapter 22, suppose that the Super implemented a __repr__ method too; would we then want to inherit Lister's or Super's? We would get it from whichever class is listed first in Sub's class header, since inheritance searches left to right. Presumably, we would list Lister first, since its whole purpose is its custom __repr__:

class Lister:
    def __repr__(self): ...

class Super:
    def __repr__(self): ...

class Sub(Lister, Super):  # Get Lister's __repr__ by listing it first.

But now suppose Super and Lister have their own versions of other same-named attributes, too. If we want one name from Super and another from Lister, no order in the class header will help—we will have to override inheritance by manually assigning to the attribute name in the Sub class:

class Lister:
    def __repr__(self): ...
    def other(self): ...

class Super:
    def __repr__(self): ...
    def other(self): ...

class Sub(Lister, Super):  # Get Lister's __repr__ by listing it first
    other = Super.other    # but explicitly pick Super's version of other.
    def __init__(self): 
        ... 

x = Sub(  )                  # Inheritance searches Sub before Super/Lister.

Here, the assignment to other within the Sub class creates Sub.other—a reference back to the Super.other object. Because it is lower in the tree, Sub.other effectively hides Lister.other, the attribute that inheritance would normally find. Similarly, if we listed Super first in the class header to pick up its other, we would then need to select Lister's method:

class Sub(Super, Lister):          # Get Super's other by order.
    __repr__ = Lister.__repr__  # Explicitly pick Lister.__repr__.

Multiple inheritance is an advanced tool. Even if you understood the last paragraph, it's still a good idea to use it sparingly and carefully. Otherwise, the meaning of a name may depend on the order in which classes are mixed in an arbitrarily far removed subclass. For another example of the technique shown here in action, see the discussion of explicit conflict resolution in Section 23.3 earlier in this chapter.

As a rule of thumb, multiple inheritance works best when your mix-in classes are as self-contained as possible—since they may be used in a variey of contexts, they should not make assumptions about the names related to other classes in a tree. Moreover, the pseudo-private attributes feature we studied earlier can help by localizing names that the a class relies on owning, and limiting the names that your mix-in classes add to the mix. In the example, if Lister only means to export its custom __repr__, it could name its other method __other to avoid clashing with other classes.

23.4.3 Class Function Attributes Are Special: Static Methods

This gotcha has been fixed by a new optional feature in Python 2.2, static and class methods, but we retain it here for readers with older Python releases, and because it gives us as good a reason as any for presenting the new static and class methods advanced feature.

In Python releases prior to 2.2, class method functions can never be called without an instance. (In Python 2.2 and later, this is also the default behavior, but it can be modified if necessary.) In the prior chapter, we talked about unbound methods: when we fetch a method function by qualifying a class (instead of an instance), we get an unbound method object. Even though they are defined with a def statement, unbound method objects are not simple functions; they cannot be called without an instance.

For example, suppose we want to use class attributes to count how many instances are generated from a class (file spam.py, shown below). Remember, class attributes are shared by all instances, so we can store the counter in the class object itself:

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1
    def printNumInstances(  ):
        print "Number of instances created: ", Spam.numInstances

But this won't work: the printNumInstances method still expects an instance to be passed in when called, because the function is associated with a class (even though there are no arguments in the def header):

>>> from spam import *
>>> a = Spam(  )
>>> b = Spam(  )
>>> c = Spam(  )
>>> Spam.printNumInstances(  )
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: unbound method must be called with class instance 1st argument

23.4.3.1 Solution (prior to 2.2, and in 2.2 normally)

Don't expect this: unbound instance methods aren't exactly the same as simple functions. This is mostly a knowledge issue, but if you want to call functions that access class members without an instance, probably the best advice is to just make them simple functions, not class methods. This way, an instance isn't expected in the call:

def printNumInstances(  ):
    print "Number of instances created: ", Spam.numInstances

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1

>>> import spam
>>> a = spam.Spam(  )
>>> b = spam.Spam(  )
>>> c = spam.Spam(  )
>>> spam.printNumInstances(  )
Number of instances created:  3
>>> spam.Spam.numInstances
3

We can also make this work by calling through an instance, as usual, although this can be inconvenient if making an instance changes the class data:

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1
    def printNumInstances(self):
        print "Number of instances created: ", Spam.numInstances

>>> from spam import Spam
>>> a, b, c = Spam(  ), Spam(  ), Spam(  )
>>> a.printNumInstances(  )
Number of instances created:  3
>>> b.printNumInstances(  )
Number of instances created:  3
>>> Spam(  ).printNumInstances(  )
Number of instances created:  4

Some language theorists claim that this means Python doesn't have class methods, only instance methods. We suspect they really mean Python classes don't work the same as in some other language. Python really has bound and unbound method objects, with well-defined semantics; qualifying a class gets you an unbound method, which is a special kind of function. Python does have class attributes, but functions in classes expect an instance argument.

Moreover, since Python already provides modules as a namespace partitioning tool, there's usually no need to package functions in classes unless they implement object behavior. Simple functions within modules usually do most of what instance-less class methods could. For example, in the first code sample in this section, printNumInstances is already associated with the class, because it lives in the same module. The only lost functionality is that the function name has a broader scope—the entire module, rather than the class.

23.4.3.2 Static and class methods in Python 2.2

As of Python 2.2, you can code classes with both static and class methods, neither of which require an instance to be present when they are invoked. To designate such methods, classes call the built-in functions staticmethod and classmethod, as hinted in the earlier discussion of new style classes. For example:

class Multi:
    def imeth(self, x):          # Normal instance method
        print self, x
    def smeth(x):                # Static: no instance passed
        print x
    def cmeth(cls, x):           # Class: gets class, not instance
        print cls, x
    smeth = staticmethod(smeth)  # Make smeth a static method.
    cmeth = classmethod(cmeth)   # Make cmeth a class method.

Notice how the last two assignments in this code simply reassign the method names smeth and cmeth. Attributes are created and changed by any assignment in a class statement, so these final assignments overwrite the assignments made earlier by the defs.

Technically, Python 2.2 supports three kinds of class-related methods: instance, static, and class. Instance methods are the normal (and default) case that we've seen in this book. With instance methods, you always must call the method with an instance object. When you call through an instance, Python passes the instance to the first (leftmost) argument automatically; when called through the class, you pass along the instance manually:

>>> obj = Multi(  )              # Make an instance
>>> obj.imeth(1)                 # Normal call, through instance
<__main__.Multi instance...> 1
>>> Multi.imeth(obj, 2)          # Normal call, through class
<__main__.Multi instance...> 2

By contrast, static methods are called without an instance argument; their names are local to the scope of the class they are defined in, and may be looked up by inheritance; mostly, they work like simple functions that happen to be coded inside a class:

>>> Multi.smeth(3)             # Static call, through class
3
>>> obj.smeth(4)               # Static call, through instance
4

Class methods are similar, but Python automatically passes the class (not an instance) in to the method's first (leftmost) argument:

>>> Multi.cmeth(5)             # Class call, through class
__main__.Multi 5
>>> obj.cmeth(6)               # Class call, through instance
__main__.Multi 6

Static and class methods are new and advanced features of the language. They have highly specialized roles that we don't have space to document here. Static methods are commonly used in conjunction with class attributes to manage information that spans all instances generated from the class.

For example, to keep track of the number of instances generated from a class (as in the earlier example), you may use static methods to manage a counter attached as a class attribute. Since such a count has nothing to do with any particular instance, it is inconvenient to have to access methods that process it through an instance (especially since making an instance to access the counter may change the counter). Moreover, static methods' proximity to the class provides a more natural solution than coding class-oriented functions outside the class. Here is the static method equivalent of this section's original example:

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances += 1
    def printNumInstances(  ):
        print "Number of instances:", Spam.numInstances
    printNumInstances = staticmethod(printNumInstances)

>>> a = Spam(  )
>>> b = Spam(  )
>>> c = Spam(  )
>>> Spam.printNumInstances(  )
Number of instances: 3
>>> a.printNumInstances(  )
Number of instances: 3

Compared to simply moving printNumInstances outside the class as prescribed earlier, this version requires an extra staticmethod call, but localizes the function name in the class scope, and moves the function code closer to where it is used (inside the class statement). You should judge for yourself whether this is a net improvement or not.

23.4.4 Methods, Classes, and Nested Scopes

This gotcha went away in Python 2.2, with the introduction of nested function scopes, but we retain it here for historical perspective, for readers working with older Python releases, and because it demonstrates what happens to the new nested function scope rules when a class is a layer of the nesting.

Classes introduce a local scope just as functions do, so the same sorts of scope behavior can happen in a class statement body. Moreover, methods are further nested functions, so the same issues apply. Confusion seems to be especially common when classes are nested.

In the following example, file nester.py, the generate function returns an instance of the nested Spam class. Within its code, the class name Spam is assigned in the generate function's local scope. But within the class's method function, the class name Spam is not visible in Python prior to 2.2 where method has access only to its own local scope, the module surrounding generate, and built-in names:

def generate(  ):
    class Spam:
        count = 1
        def method(self):        # Name Spam not visible:
            print Spam.count     # not local(def),global(module), built-in
    return Spam(  )

generate(  ).method(  )

C:\python\examples> python nester.py
Traceback (innermost last):
  File "nester.py", line 8, in ?
    generate(  ).method(  )
  File "nester.py", line 5, in method
    print Spam.count             # Not local(def),global(module), built-in
NameError: Spam

As a solution, either upgrade to Python 2.2, or don't nest code this way. This example works in Python 2.2 and later, because the local scopes of all enclosing function defs are automatically visible to nested defs, including nested method defs, as in this example.

Note that even in 2.2, method defs cannot see the local scope of the enclosing class, only the local scope of enclosing defs. That's why methods must go through the self instance or the class name, to reference methods and other attributes defined in the enclosing class statement. For example, code in the method must use self.count or Spam.count, not just count.

Prior to release 2.2, there are a variety of ways to get the example above to work. One of the simplest is to move the name Spam out to the enclosing module's scope with global declarations; since method sees global names in the enclosing module, references work:

def generate(  ):
    global Spam                 # Force Spam to module scope.
    class Spam:
        count = 1
        def method(self):
            print Spam.count        # Works: in global (enclosing module)
    return Spam(  )

generate(  ).method(  )             # Prints 1

Perhaps better, we can also restructure the code such that class Spam is defined at the top level of the module by virtue of its nesting level, rather than global declarations. Both the nested method function and the top level generate find Spam in their global scopes:

def generate(  ):
    return Spam(  )

class Spam:                    # Define at module top-level.
    count = 1
    def method(self):
        print Spam.count       # Works: in global (enclosing module)

generate(  ).method(  )

In fact, this is what we prescribe for all Python releases—your code tends to be simpler in general if you avoid nesting of classes and functions.

If you want to get complicated and tricky, you can also get rid of the Spam reference in method altogether, by using the special __class__ attribute, which returns an instance's class object:

def generate(  ):
    class Spam:
        count = 1
        def method(self):
            print self.__class__.count       # Works: qualify to get class
    return Spam(  )

generate( ).method( )

Why You Will Care: OOP by the Masters When I (Mark) teach Python classes, invariably, about halfway through the class, people who have used OOP in the past are following along intensely; people who have not are beginning to glaze over (or nod off completely). The point behind the technology just isn't apparent. In a book like this, we have the luxury of adding overview material like the new "Big Picture" overview in Chapter 19, and you should probably review that section if you're starting to feel like OOP is just some computer science mumbo-jumbo. In real classes, however, to help get the newcomers on board (and awake), I have been known to stop and ask the experts in the audience why they use OOP at all. The answers they've given might help shed some light on the purpose of OOP, if you are new to the subject. Here, then, with only a few embellishments, are the most common reasons to use OOP, as cited by my students over the years: Code reuse This one's easy (and is the main reason for using OOP). By supporting inheritance, classes allow you to program by customization, instead of starting each project from scratch. Encapsulation By wrapping up implementation details behind object interfaces, users of a class are insulated from code changes. Structure Classes provide a new local scope, which minimizes name clashes. They also provide a natural place to write and look for implementation code, and manage object state. Maintenance Thanks both to the structure and code reuse support of classes, there is usually only one copy of code to be changed. Consistency Classes and inheritance allow you to implement common interfaces, and hence a common look-and-feel in your code; this eases debugging, comprehension, and maintenance. Polymorphism This is more a property of OOP than a reason; but by supporting generality of code, polymorphism makes that code more flexible and widely applicable, and hence more reusable. And of course, the number-one reason students gave for using OOP: it looks good on a resume. Finally, keep in mind what we said at the beginning of Part VI: you won't fully appreciate OOP until you've used it for a while. Pick a project, study larger examples, work through the exercises—whatever it takes to get your feet wet with OO code; it's worth the effort.

23.4.5 Overwrapping-itis

Sometimes, the abstraction potential of OOP can be abused to the point of making code difficult to understand. If your classes are layered too deeply, it can make code obscure; you may have to search through many classes to discover what an operation does. For example, one of your authors once worked in a C++ shop with thousands of classes (some generated by machine), and up to 15 levels of inheritance; deciphering a method call in such a complex system was often a monumental task. Multiple classes had to be consulted for even the most basic of operations.

The most general rule of thumb applies here too: don't make things complicated unless they truly must be. Wrapping your code in multiple layers of classes to the point of incomprehensibility is always a bad idea. Abstraction is the basis of polymorphism and encapsulation, and can be a very effective tool when used well. But you'll simplify debugging and aid maintainability, if you make your class interfaces intuitive, avoid making code overly abstract, and keep your class hierarchies short and flat unless there is a good reason to do otherwise.