14.8 Function Gotchas

Here are some of the more jagged edges of functions you might not expect. They're all obscure, and a few have started to fall away from the language completely in recent releases, but most have been known to trip up a new user.

14.8.1 Local Names Are Detected Statically

Python classifies names assigned in a function as locals by default; they live in the function's scope and exist only while the function is running. What we didn't tell you is that Python detects locals statically, when it compiles the def's code, rather than by noticing assignments as they happen at runtime. This leads to one of the most common oddities posted on the Python newsgroup by beginners.

Normally, a name that isn't assigned in a function is looked up in the enclosing module:

>>> X = 99
>>> def selector(  ):        # X used but not assigned
...     print X              # X found in global scope
...
>>> selector(  )
99

Here, the X in the function resolves to the X in the module outside. But watch what happens if you add an assignment to X after the reference:

>>> def selector(  ):
...     print X              # Does not yet exist!
...     X = 88               # X classified as a local name (everywhere)
...                          # Can also happen if "import X", "def X",...
>>> selector(  )
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 2, in selector
UnboundLocalError: local variable 'X' referenced before assignment

You get an undefined name error, but the reason is subtle. Python reads and compiles this code when it's typed interactively or imported from a module. While compiling, Python sees the assignment to X and decides that X will be a local name everywhere in the function. But later, when the function is actually run, the assignment hasn't yet happened when the print executes, so Python says you're using an undefined name. According to its name rules, it should; local X is used before being assigned. In fact, any assignment in a function body makes a name local. Imports, =, nested defs, nested classes, and so on, are all susceptible to this behavior.

The problem occurs because assigned names are treated as locals everywhere in a function, not just after statements where they are assigned. Really, the previous example is ambiguous at best: did you mean to print the global X and then create a local X, or is this a genuine programming error? Since Python treats X as a local everywhere, it is an error; but if you really mean to print global X, you need to declare it in a global statement:

>>> def selector(  ):
...     global X           # Force X to be global (everywhere).
...     print X
...     X = 88
...
>>> selector(  )
99

Remember, though, that this means the assignment also changes the global X, not a local X. Within a function, you can't use both local and global versions of the same simple name. If you really meant to print the global and then set a local of the same name, import the enclosing module and qualify to get to the global version:

>>> X = 99
>>> def selector(  ):
...     import __main__     # Import enclosing module.
...     print __main__.X    # Qualify to get to global version of name.
...     X = 88                  # Unqualified X classified as local.
...     print X                 # Prints local version of name.
...
>>> selector(  )
99
88

Qualification (the .X part) fetches a value from a namespace object. The interactive namespace is a module called __main__, so __main__.X reaches the global version of X. If that isn't clear, check out Part V.^[5]

^[5] Python has improved on this story somewhat, by issuing the more specific "unbound local" error message for this case shown in the example listing (it used to simply raise a generic name error); this gotcha is still present in general, though.

14.8.2 Defaults and Mutable Objects

Default argument values are evaluated and saved when the def statement is run, not when the resulting function is called. Internally, Python saves one object per default argument, attached to the function itself.

That's usually what you want; because defaults are evaluated at def time, it lets you save values from the enclosing scope if needed. But since defaults retain an object between calls, you have to be careful about changing mutable defaults. For instance, the following function uses an empty list as a default value and then changes it in place each time the function is called:

>>> def saver(x=[  ]):            # Saves away a list object
...     x.append(1)             # Changes same object each time!
...     print x
...
>>> saver([2])                  # Default not used
[2, 1]
>>> saver(  )                   # Default used
[1]
>>> saver(  )                   # Grows on each call!
[1, 1]
>>> saver(  )
[1, 1, 1]

Some see this behavior as a feature—because mutable default arguments retain their state between function calls, they can serve some of the same roles as static local function variables in the C language. In a sense, they work something like global variables, but their names are local to the function, and so will not clash with names elsewhere in a program.

To most observers, though, this seems like a gotcha, especially the first time they run into this. There are better ways to retain state between calls in Python (e.g., using classes, which will be discussed in Part VI).

Moreover, mutable defaults are tricky to remember (and understand at all). They depend upon the timing of default object construction. In the example, there is just one list object for the default value—the one created when the def was executed. You don't get a new list every time the function is called, so the list grows with each new append; it is not reset to empty on each call.

If that's not the behavior you wish, simply make copies of the default at the start of the function body, or move the default value expression into the function body; as long as the value resides in code that's actually executed each time the function runs, you'll get a new object each time through:

>>> def saver(x=None):
...     if x is None:         # No argument passed?
...         x = [  ]            # Run code to make a new list.
...     x.append(1)           # Changes new list object
...     print x
...
>>> saver([2])
[2, 1]
>>> saver(  )                   # Doesn't grow here
[1]
>>> saver(  )
[1]

By the way, the if statement in the example could almost be replaced by the assignment x = x or [ ], which takes advantage of the fact that Python's or returns one of its operand objects: if no argument was passed, x defaults to None, so the or returns the new empty list on the right.

However, this isn't exactly the same. When an empty list is passed in, the or expression would cause the function to extend and return a newly created list, rather than extending and returning the passed-in list like the previous version. (The expression becomes [ ] or [ ], which evaluates to the new empty list on the right; see Section 9.3 in Chapter 9 if you don't recall why). Real program requirements may call for either behavior.

14.8.3 Functions Without Returns

In Python functions, return (and yield) statements are optional. When a function doesn't return a value explicitly, the function exits when control falls off the end of the function body. Technically, all functions return a value; if you don't provide a return, your function returns the None object automatically:

>>> def proc(x):
...     print x        # No return is a None return.
...
>>> x = proc('testing 123...')
testing 123...
>>> print x
None

Functions such as this without a return are Python's equivalent of what are called "procedures" in some languages. They're usually invoked as a statement, and the None result is ignored, since they do their business without computing a useful result.

This is worth knowing, because Python won't tell you if you try to use the result of a function that doesn't return one. For instance, assigning the result of a list append method won't raise an error, but you'll really get back None, not the modified list:

>>> list = [1, 2, 3]
>>> list = list.append(4)      # append is a "procedure."
>>> print list                 # append changes list in-place. 
None

As mentioned in Section 11.2 in Chapter 11, such functions do their business as a side effect, and are usually designed to be run as a statement, not an expression.