13.3 Scopes and Nested Functions

It's time to take a deeper look at the letter "E" in the LEGB lookup rule. The "E" layer takes the form of the local scopes of any and all enclosing function defs. This layer is a relatively new addition to Python (added in Python 2.2), and is sometimes called statically nested scopes. Really, the nesting is a lexical one—nested scopes correspond to physically nested code structures in your program's source code.

13.3.1 Nested Scope Details

With the addition of nested function scopes, variable lookup rules become slightly more complex. Within a function:

Assignment: X=value

Creates or changes name X in the current local scope by default. If X is declared global within the function, it creates or changes name X in the enclosing module's scope instead.

Reference: X

Looks for name X in the current local scope (function), then in the local scopes of all lexically enclosing functions from inner to outer (if any), then in the current global scope (the module file), and finally in the built-in scope (module __builtin__). global declarations make the search begin in the global scope instead.

Notice that the global declaration still maps variables to the enclosing module. When nested functions are present, variables in enclosing functions may only be referenced, not changed. Let's illustrate all this with some real code.

13.3.2 Nested Scope Examples

Here is an example of a nested scope:

def f1(  ):
    x = 88
    def f2(  ):
        print x
    f2(  )

f1(  )                  # Prints 88

First off, this is legal Python code: the def is simply an executable statement that can appear anywhere any other statement can—including nested in another def. Here, the nested def runs while a call to function f1 is running; it generates a function and assigns it to name f2, a local variable within f1's local scope. In a sense, f2 is a temporary function, that only lives during the execution of (and is only visible to code in) the enclosing f1.

But notice what happens inside f2: when it prints variable x, it refers to the x that lives in the enclosing f1 function's local scope. Because functions can access names in all physically enclosing def statements, the x in f2 is automatically mapped to the x in f1, by the LEGB lookup rule.

This enclosing scope lookup works even if the enclosing function has already returned. For example, the following code defines a function that makes and returns another:

def f1(  ):
    x = 88
    def f2(  ):
        print x
    return f2

action = f1(  )            # Make, return function.
action(  )                 # Call it now: prints 88

In this code, the call to action is really running the function we named f2 when f1 ran. f2 remembers the enclosing scope's x in f1, even though f1 is no longer active. This sort of behavior is also sometimes called a closure—an object that remembers values in enclosing scopes, even though those scopes may not be around any more. Although classes (described in Part VI) are usually best at remembering state, such functions provide another alternative.

In earlier versions of Python, all this sort of code failed, because nested defs did not do anything about scopes—a reference to a variable within f2 would search local (f2), then global (the code outside f1), and then built-in. Because it skipped the scopes of enclosing functions, an error would result. To work around this, programmers typically used default argument values to pass-in (remember) the objects in an enclosing scope:

def f1(  ):
    x = 88
    def f2(x=x):
        print x
    f2(  )

f1(  )                  # Prints 88

This code works in all Python releases, and you'll still see this pattern in much existing Python code. We'll meet defaults in more detail later in this chapter; in short, the syntax arg=val in a def header means that argument arg will default to value val, if no real value is passed to arg in a call.

In the modified f2, the x=x means that argument x will default to the value of x in the enclosing scope—because the second x is evaluated before Python steps into the nested def, it still refers to the x in f1. In effect, the default remembers what x was in f1, the object 88.

That's fairly complex, and depends entirely on the timing of default value evaluations. That's also why the nested scope lookup rule was added to Python, to make defaults unnecessary for this role. Today, Python automatically remembers any values required in the enclosing scope, for use in nested defs.

Of course, the best prescription here is probably just don't do that. Programs are much simpler if you do not nest defs within defs. Here's an equivalent of the prior example, which banishes the notion of nesting; notice that it's okay to call a function (see following) defined after the one that contains the call like this, as long as the second def runs before the call of the first function—code inside a def is never evaluated until the function is actually called:

>>> def f1(  ):
...     x = 88
...     f2(x)
...
>>> def f2(x):
...     print x
...
>>> f1(  )
88

If you avoid nesting this way, you can almost forget about the nested scopes concept in Python, at least for defs. However, you are even more likely to care about such things when you start coding lambda expressions. We also won't meet lambda in depth until Chapter 14; in short, lambda is an expression that generates a new function to be called later, much like a def statement (because it's an expression, it can be used in places that def cannot, such as within list and dictionary literals).

Also like a def, lambda expressions introduce a new local scope. With the enclosing scopes lookup layer, they can see all the variables that live in the function in which they are coded. The following works today, only because the nested scope rules are now applied:

def func(  ):
    x = 4
    action = (lambda n: x ** n)          # x in enclosing def
    return action

x = func(  )
print x(2) # Prints 16

Prior to the introduction of nested function scopes, programmers used defaults to pass values from an enclosing scope into lambdas, as for defs. For instance, the following works on all Python releases:

def func(  ):
    x = 4
    action = (lambda n, x=x: x ** n)     # Pass x in manually.

Because lambdas are expressions, they naturally (and even normally) nest inside enclosing defs. Hence, they are perhaps the biggest beneficiary of the addition of enclosing function scopes in the lookup rules; in most cases, it is no longer necesary to pass values into lambdas with defaults. We'll have more to say about both defaults and lambdas later, so you may want to return and review this section later.

Before ending this discussion, note that scopes nest arbitrarily, but only enclosing functions (not classes, described in Part VI) are searched:

>>> def f1(  ):
...     x = 99
...     def f2(  ):
...         def f3(  ):
...             print x       # Found in f1's local scope!
...         f3(  )
...     f2(  )
...
>>> f1(  )
99

Python will search the local scopes of all enclosing defs, after the referencing function's local scope, and before the module's global scope. However, this sort of code seems less likely in practice.