16.3 Module Namespaces

Modules are probably best understood as simply packages of names—places to define names you want to make visible to the rest of a system. In Python, modules are a namespace—a place where names are created. Names that live in a module are called its attributes. Technically, modules usually correspond to files, and Python creates a module object to contain all the names assigned in the file; but in simple terms, modules are just namespaces.

16.3.1 Files Generate Namespaces

So how do files morph into namespaces? The short story is that every name that is assigned a value at the top level of a module file (i.e., not nested in a function or class body) becomes an attribute of that module.

For instance, given an assignment statement such as X=1 at the top level of a module file M.py, the name X becomes an attribute of M, which we can refer to from outside the module as M.X. The name X also becomes a global variable to other code inside M.py, but we need to explain the notion of module loading and scopes a bit more formally to understand why:

• Module statements run on the first import. The first time a module is imported anywhere in a system, Python creates an empty module object and executes the statements in the module file one after another, from the top of the file to the bottom.

• Top-level assignments create module attributes. During an import, statements at the top-level of the file that assign names (e.g., =, def) create attributes of the module object; assigned names are stored in the module's namespace.

• Module namespace: attribute __dict__, or dir(M). Module namespaces created by imports are dictionaries; they may be accessed through the built-in __dict__ attribute associated with module objects and may be inspected with the dir function. The dir function is roughly equivalent to the sorted keys list of an object's __dict__ attribute, but includes inherited names for classes, may not be complete, and is prone to change from release to release.

• Modules are a single scope (local is global). As we saw in Chapter 13, names at the top level of a module follow the same reference/assignment rules as names in a function, but the local and global scopes are the same (or, more accurately, it's the LEGB rule, but without the L and E lookup layers). But in modules, the module scope becomes an attribute dictionary of a module object, after the module has been loaded. Unlike functions (where the local namespace exists only while the function runs), a module file's scope becomes a module object's attribute namespace and lives on after the import.

Here's a demonstration of these ideas. Suppose we create the following module file with a text editor and call it module2.py:

print 'starting to load...'

import sys
name = 42

def func(  ): pass

class klass: pass

print 'done loading.'

The first time this module is imported (or run as a program), Python executes its statements from top to bottom. Some statements create names in the module's namespace as a side effect, but others may do actual work while the import is going on. For instance, the two print statements in this file execute at import time:

>>> import module2
starting to load...
done loading.

But once the module is loaded, its scope becomes an attribute namespace in the module object we get back from import—we access attributes in this namespace by qualifying them with the name of the enclosing module:

>>> module2.sys
<module 'sys'>
>>> module2.name
42
>>> module2.func, module2.klass
(<function func at 765f20>, <class klass at 76df60>)

Here, sys, name, func, and klass were all assigned while the module's statements were being run, so they're attributes after the import. We'll talk about classes in Part VI, but notice the sys attribute; import statements really assign module objects to names and any type of assignment to a name at the top level of a file generates a module attribute. Internally, module namespaces are stored as dictionary objects. In fact, we can access the namespace dictionary through the module's __dict__ attribute; it's just a normal dictionary object, with the usual methods:

>>> module2.__dict__.keys(  )
['__file__', 'name', '__name__', 'sys', '__doc__', '__builtins__', 'klass', 
'func']

The names we assigned in the module file become dictionary keys internally. Some of the names in the module's namespace are things Python adds for us; for instance, __file__ gives the name of the file the module was loaded from, and __name__ gives its name as known to importers (without the .py extension and directory path).

16.3.2 Attribute Name Qualification

Now that you're becoming more familiar with modules, we should clarify the notion of name qualification. In Python, you can access attributes in any object that has attributes, using the qualification syntax object.attribute.

Qualification is really an expression that returns the value assigned to an attribute name associated with an object. For example, the expression module2.sys in the previous example fetches the value assigned to sys in module2. Similarly, if we have a built-in list object L, L.append returns the method associated with the list.

So what does attribute qualification do to the scope rules we studied in Chapter 13? Nothing, really: it's an independent concept. When you use qualification to access names, you give Python an explicit object to fetch from. The LEGB rule applies only to bare, unqualified names. Here are the rules:

Simple variables

X means search for name X in the current scopes (LEGB rule).

Qualification

X.Y means find X in the current scopes, then search for attribute Y in the object X (not in scopes).

Qualification paths

X.Y.Z means look up name Y in object X, then look up Z in object X.Y.

Generality

Qualification works on all objects with attributes: modules, classes, C types, etc.

In Part VI, we'll see that qualification means a bit more for classes (it's also the place where something called inheritance happens), but in general, the rules here apply to all names in Python.

16.3.3 Imports Versus Scopes

It is never possible to access names defined in another module file without first importing that file. That is, you never automatically get to see names in another file, regardless of the structure of imports or function calls in your program.

For example, consider the following two simple modules. The first, moda.py, defines a variable X global to code in its file only, along with a function that changes the global X in this file:

X = 88               # My X: global to this file only

def f(  ):
    global X         # Change my X.
    X = 99           # Cannot see names in other modules

The second module, modb.py, defines its own global variable X, and imports and calls the function in the first module:

X = 11                 # My X: global to this file only

import moda            # Gain access to names in moda.
moda.f(  )             # Sets moda.X, not my X
print X, moda.X

When run, moda.f changes the X in moda, not the X in modb. The global scope for moda.f is always the file enclosing it, regardless of which module it is ultimately called from:

% python modb.py
11 99

In other words, import operations never give upward visibility to code in imported files—it cannot see names in the importing file. More formally:

• Functions can never see names in other functions, unless they are physically enclosing.

• Module code can never see names in other modules unless they are explicitly imported.

Such behavior is part of the lexical scoping notion—in Python, the scopes surrounding a piece of code are completely determined from the code's physical position in your file. Scopes are never influenced by function calls, or module imports.^[1]

^[1] Some languages act differently and provide for dynamic scoping, where scopes really may depend on runtime calls. This tends to make code trickier, though, because the meaning of a variable can differ over time.

16.3.4 Namespace Nesting

In some sense, although imports do not nest namespaces upward, they do nest downward. Using attribute qualification paths, it's possible to descend into arbitrarily nested modules, and access their attributes. For example, consider the next three files. mod3.py defines a single global name and attribute by assignment:

X = 3

mod2.py imports the first and uses qualification to access the imported module's attribute:

X = 2
import mod3

print X,                # My global X
print mod3.X            # mod3's X

And mod1.py imports the second, and fetches attributes in both the first and second files:

X = 1
import mod2

print X,                    # My global X
print mod2.X,               # mod2's X
print mod2.mod3.X           # Nested mod3's X

Really, when mod1 imports mod2 here, it sets up a two-level namespace nesting. By using a path of names mod2.mod3.X, it descends into mod3, which is nested in the imported mod2. The net effect is that mod1 can see the Xs in all three files, and hence has access to all three global scopes:

% python mod1.py
2 3
1 2 3

Conversely, mod3 cannot see names in mod2, and mod2 cannot see names in mod1. This example may be easier to grasp if you don't think in terms of namespaces and scopes; instead, focus on the objects involved. Within mod1, mod2 is just a name that refers to an object with attributes, some of which may refer to other objects with attributes (import is an assignment). For paths like mod2.mod3.X, Python simply evaluates left to right, fetching attributes from objects along the way.

Note that mod1 can say import mod2 and then mod2.mod3.X, but cannot say import mod2.mod3—this syntax invokes something called package (directory) imports, described in the next chapter. Package imports also create module namespace nesting, but their import statements are taken to reflect directory trees, not simple import chains.