28.1 An Automated Complaint System

The scenario for this example is that of a startup company, Joe's Toothpaste, Inc., which sells the latest in 100% organic, cruelty-free, tofu-based toothpaste. Since there is only one employee, and that employee is quite busy shopping for the best tofu he can find, the tube doesn't say "For customer complaints or comments, call 1-800-TOFTOOT," but instead, says, "If you have a complaint or wish to make a comment, visit our web site at www.toftoot.com." The web site has all the usual glossy pictures and an area where the customer can enter a complaint or comment. This page looks like Figure 28-1.

Figure 28-1. What the customer finds at http://www.toftoot.com/comment.html

28.1.1 Excerpt from the HTML File

The key parts of the HTML code that generated this page are:

<form method="post" action="http://toftoot.com/cgi-bin/feedback.py">
<ul><i>Please fill out the entire form:</i></ul>
<center><table width="100%" >
<tr>
    <td align="right" width="20%">Name:</td>
    <td>
        <input type="text" name="name" size="50" value="">
    </td>
</tr>
<tr>
    <td align="right">Email Address:</td>
    <td>
        <input type="text" name="email" size="50" value="">
    </td>
</tr>
<tr>
    <td align="right">Mailing Address:</td>
    <td>
       <input type="text" name="address" size="50" value="">
    </td>
</tr>
<tr>
    <td align="right">Type of Message:</td>
    <td>
        <input type="radio" name="type" checked 
               value="comment">comment&nbsp;</input>
        <input type="radio" name="type" 
               value="complaint">complaint</input>
    </td>
</tr>
<tr>
    <td align="right" valign="top">
         Enter the text in here:</td>
    <td><textarea name="text" rows="5" cols="50" value="">
        </textarea></td></tr>
<tr>
    <td></td>
    <td>
    <input type="submit" name="send" value="Send the feedback!">
    </td>
</tr>
</table></center>
</form>

We assume that you know enough about CGI and HTML to follow this discussion. The HTML code generates the web page shown in Figure 28-1:

• The form line specifies what CGI program should be invoked when the form is submitted; specifically, the URL points to a script called feedback.py.

• The input tags indicate the names of the fields in the form (name, address, email, and text, as well as type). The values of these fields are whatever the user enters, except for the type, which takes either the value 'comment' or 'complaint', depending on which radio button the user selected.

• The input type="submit" tag is for the submission button, which actually calls the CGI script.

We now get to the interesting part as far as Python is concerned: the processing of the request. Here is the entire feedback.py program:

1        #!c:/python23/python.exe
2        import cgi, cgitb, os, sys, string, time
3        cgitb.enable(  )

4        def gush(data):
5            print """Content-type: text/html\n
6        <h3>Thanks, %(name)s!</h3>
7        Our customer's comments are always appreciated.
8        They drive our business directions, as well as
9        help us with our karma.
10        <p>Thanks again for the feedback!<p>
11        And feel free to enter more comments if you wish.""" % vars(data)
12            print "<p>"+10*"&nbsp;"+"--Joe."
13        
14        def whimper(data):
15            print """Content-type: text/html\n
16        <h3>Sorry, %(name)s!</h3>
17        We're very sorry to read that you had a complaint"
18        regarding our product__We'll read your comments"
19        carefully and will be in touch with you."
20        <p>Nevertheless, thanks for the feedback.<p>""" % vars(data)
21            print "<p>"+10*"&nbsp;"+"--Joe."
        
22        def bail(  ):
23            print """<h3>Error filling out form</h3>
24        Please fill in all the fields in the form.<p>
25        <a href="http://localhost/comment.html">
26        Go back to the form</a>"""
27            sys.exit(  )

28        class FormData:
29            """ A repository for information gleaned from a CGI form """
30            def __init__(self, form):
31                self.time = time.asctime(  )
32                for fieldname in self.fieldnames:
33                    if fieldname not in form or form[fieldname].value == "":
34                        bail(  )
35                    else:
36                        setattr(self, fieldname, form[fieldname].value)
                
37        class FeedbackData(FormData):
38            """ A FormData generated by the comment.html form. """
39            fieldnames = ('name', 'address', 'email', 'type', 'text')
40            def __repr__(self):
41                return "%(type)s from %(name)s on %(time)s" % vars(self)
        
42        DIRECTORY = r'C:\complaintdir'

43        if __name__ == '__main__':
44            sys.stderr = sys.stdout 
45            form = cgi.FieldStorage(  )
46            data = FeedbackData(form)
47            if data.type == 'comment':
48                gush(data)
49            else:
50                whimper(data)

51            # Save the data to file.
52            import tempfile, pickle
53            tempfile.tempdir = DIRECTORY
54            pickle.dump(data, open(tempfile.mktemp(  ), 'w'))

The output of this script clearly depends on the input, but the output with the form filled out with the parameters shown in Figure 28-1 is displayed in Figure 28-2.

Figure 28-2. What the user sees after pressing the Send the feedback button

How does the feedback.py script work? There are a few aspects of the script common to all CGI programs, and those are highlighted in bold. To start, the first line of the program needs to refer to the Python executable. This is a requirement of the web server we're using here, and it might not apply in your case; even if it does, the specific location of your Python program is likely to be different from this. The second line includes imports of cgi and cgitb. The cgi module deals with the hard parts of CGI, such as parsing the environment variables and handling escaped characters. The cgitb module stands for "CGI Traceback," and it makes debugging CGI applications much easier. It needs to be enabled (line 3) to turn exceptions in the script into prettily-printed tracebacks. The documentation for the cgi module describes a very straightforward and easy way to use it. For this example, however, mostly because we're going to build on it, the script is somewhat more complicated than strictly necessary.

Let's just go through the code in the if __name__ == '__main__' block one statement at a time.^[1] The first statement (line 44) redirects the sys.stderr stream to whatever standard output is. This is done for debugging because the output of the stdout stream in a CGI program goes back to the web browser, and the stderr stream goes to the server's error log, which can be harder to read than simply looking at the web page. This way, if a runtime exception occurs, we can see it on the web page, as opposed to having to guess what it was.

^[1] You'll remember that this if statement is true only when the program is run as a script, not when it's imported. CGI programs qualify as scripts, so the code in the if block runs when this program is called by the web server. We use it later as an imported script.

The second line (line 45) is crucial and does all of the hard CGI work: it returns a dictionary-like object (called a FieldStorage object) whose keys are the names of the variables filled out in the form, while the value of each field in the form can be obtained by asking for the value attribute of the entries in the FieldStorage object. Sounds complicated, but all it means is that for our form, the form object has keys 'name', 'type', 'email', 'address', and 'text', and that to find out what the user entered in the Name field of the web form, we need to look at form['name'].value.

The third line in the if block (line 46) creates an instance of our user-defined class FeedbackData, passing the form object as an argument. If you now look at the definition of the FeedbackData class, you'll see that it's a very simple subclass of FormData, which is also a user-defined class. All we've defined in the FeedbackData subclass is a class attribute fieldnames and a __repr__ function (used by the print statement, among others). Clearly, the __init__ method of the FormData class must do something with the FieldStorage argument. Indeed, it looks at each of the field names defined in the fieldnames class attribute of the instance (that's what self.fieldnames refers to), and for each field name, checks whether the FieldStorage object has a corresponding nonempty key. If it does, it sets an attribute with the same name as the field in the instance, giving it as value the text entered by the user. If it doesn't, it calls the bail function.

Now, let's walk through the usual case, when the user dutifully enters all of the required data. In those cases, FieldStorage has all of the keys ('name', 'type', etc.), which the FeedbackData class needs. The FormData class' __init__ method in turn sets attributes for each field name in the instance. So, when the data = FeedbackData(form) call returns, data is guaranteed to be an instance of FeedbackData, which is a subclass of FormData, and data has the attributes name, type, email, etc., with the corresponding values the user entered. In addition, the instance will also have an attribute (time) corresponding to the time at which the instance was created.

A similar effect could have been obtained with code like:

form = cgi.FieldStorage(  )
form_ok = 1
if 'name' not in form or form["name"].value == "":
    form_ok = 0
else:
    data_name = form["name"].value
if 'email' not in form or form["email"].value == "":
    form_ok = 0
else:
    data_email = form["email"].value
...

but it should be clear that this kind of programming can get very tedious, repetitive, and error-prone. With our scheme, when Joe changes the set of field names in the web page, all we need to change is the fieldnames attribute of the FeedbackData class. Also, we can use the same FormData class in any other CGI script, and thus reuse code.

What if the user didn't enter all of the required fields? Either the FieldStorage dictionary will be missing a key, or its value will be the empty string. The FormData.__init__ method then calls the bail function, which displays a polite error message and exits the script. Control never returns back to the main function, so there is no need to test the validity of the data variable; if we got something back from FeedbackData( ), it's a valid instance.

With the data instance, we check to see if the feedback type was a comment, in which case we thank the user for their input. If the feedback type was a complaint, we apologize profusely and promise to get back in touch with them.

We now have a basic CGI infrastructure in place. To save the data to file is remarkably easy:

• First, we define the DIRECTORY variable outside the if test because we'll use it from another script that will import this one, so we wish it to be defined even if this script is not run as a program.

• At line 52, we import the tempfile and pickle modules. The tempfile module comes up with filenames currently not in use; that way we don't need to worry about "collisions" in any filename generation scheme. The pickle module will allow us to serialize (i.e., save) any Python object.

• Line 53: The next line sets the tempdir attribute of the tempfile module to the value of the DIRECTORY variable, which is where we want our data to be saved. This is an example of customizing an existing module by directly modifying its namespace, just as we modified the stderr attribute of the sys module earlier.

• Line 54: The last line does the actual saving; it opens in write mode the file with a name generated by the tempfile module and dumps the instance data into it, using the pickle module. The pickle module is one of the gems that makes using a high-level language like Python so productive—pickling means taking arbitrary Python objects and converting them into streams of bytes in a format that Python knows how to "unpickle." Since they're streams of bytes, they can be written to disk, as we're doing here, or sent over a network, or otherwise stored or transmitted for later unpickling. There was no need to come with a file format specific to this application. As you can see on line 54, all it takes to save a Python object is to pass it to pickle.dump( ), take the result and stuff it in a file. Now the specified file contains a pickled instance, which we'll unpickle in the next section.