18.6 Module Design Concepts

Like functions, modules present design tradeoffs: deciding which functions go in which module, module communication mechanisms, and so on. Here are a few general ideas that will become clearer when you start writing bigger Python systems:

• You're always in a module in Python. There's no way to write code that doesn't live in some module. In fact, code typed at the interactive prompt really goes in a built-in module called __main__; the only unique things about the interactive prompt is that code runs and is disgarded immediately, and that expression results are printed.

• Minimize module coupling: global variables. Like functions, modules work best if they're written to be closed boxes. As a rule of thumb, they should be as independent of global names in other modules as possible.

• Maximize module cohesion: unified purpose. You can minimize a module's couplings by maximizing its cohesion; if all the components of a module share its general purpose, you're less likely to depend on external names.

• Modules should rarely change other modules' variables. It's perfectly okay to use globals defined in another module (that's how clients import services), but changing globals in another module is often a symptom of a design problem. There are exceptions of course, but you should try to communicate results through devices such as function return values, not cross-module changes. Otherwise your globals' values become dependent on the order of arbitrarily remote assignments.

As a summary, Figure 18-1 sketches the environment in which modules operate. Modules contain variables, functions, classes, and other modules (if imported). Functions have local variables of their own. You'll meet classes—another object that lives within modules—in Chapter 19.

Figure 18-1. Module environment

18.6.1 Modules Are Objects: Metaprograms

Because modules expose most of their interesting properties as built-in attributes, it's easy to write programs that manage other programs. We usually call such manager programs metaprograms, because they work on top of other systems. This is also referred to as introspection, because programs can see and process object internals. Introspection is an advanced feature, but can be useful for building programming tools.

For instance, to get to an attribute called name in a module called M, we can either use qualification, or index the module's attribute dictionary exposed in the built-in _ _dict__ attribute. Further, Python also exports the list of all loaded modules as the sys.modules dictionary (that is, the modules attribute of the sys module), and provides a built-in called getattr that lets us fetch attributes from their string names (it's like saying object.attr, but attr is a runtime string). Because of that, all the following expressions reach the same attribute and object:

M.name                          # Qualify object.
M.__dict__['name']               # Index namespace dictionary manually.
sys.modules['M'].name           # Index loaded-modules table manually.
getattr(M, 'name')              # Call built-in fetch function.

By exposing module internals like this, Python helps you build programs about programs.^[1] For example, here is a module named mydir.py that puts these ideas to work, to implement a customized version of the built-in dir function. It defines and exports a function called listing, which takes a module object as an argument and prints a formatted listing of the module's namespace:

^[1] Notice that because a function can access its enclosing module by going through the sys.modules table like this, it's possible to emulate the effect of the global statement you met in Chapter 13. For instance, the effect of global X; X=0 can be simulated by saying, inside a function: import sys; glob=sys.modules[__name__]; glob.X=0 (albeit with much more typing). Remember, each module gets a __name__ attribute for free; it's visible as a global name inside functions within a module. This trick provides another way to change both local and global variables of the same name, inside a function.

# A module that lists the namespaces of other modules

verbose = 1

def listing(module):
    if verbose:
        print "-"*30
        print "name:", module.__name__, "file:", module.__file__
        print "-"*30

    count = 0
    for attr in module.__dict__.keys(  ):      # Scan namespace.
        print "%02d) %s" % (count, attr),
        if attr[0:2] == "__":
            print "<built-in name>"           # Skip __file__, etc.
        else:
            print getattr(module, attr)       # Same as .__dict__[attr]
        count = count+1

    if verbose:
        print "-"*30
        print module.__name__, "has %d names" % count
        print "-"*30

if __name__ == "__main__":
    import mydir
    listing(mydir)      # Self-test code: list myself

We've also provided self-test logic at the bottom of this module, which narcissistically imports and lists itself. Here's the sort of output produced:

C:\python> python mydir.py
------------------------------
name: mydir file: mydir.py
------------------------------
00) __file__ <built-in name>
01) __name__ <built-in name>
02) listing <function listing at 885450>
03) __doc__ <built-in name>
04) __builtins__ <built-in name>
05) verbose 1
------------------------------
mydir has 6 names
------------------------------

We'll meet getattr and its relatives again. The point to notice here is that mydir is a program that lets you browse other programs. Because Python exposes its internals, you can process objects generically.^[2]

^[2] Tools such as mydir.listing can be preloaded into the interactive namespace, by importing them in the file referenced by the PYTHONSTARTUP environment variable. Since code in the startup file runs in the interactive namespace (module __main__), imports of common tools in the startup file can save you some typing. See Appendix A for more details.