10.4 Loop Variations

The for loop subsumes most counter-style loops. It's generally simpler to code and quicker to run than a while, so it's the first tool you should reach for whenever you need to step through a sequence. But there are also situations where you will need to iterate in a more specialized way. For example, what if you need to visit every second or third item in a list, or change the list along the way? How about traversing more than one sequence in parallel, in the same for loop?

You can always code such unique iterations with a while loop and manual indexing, but Python provides two built-ins that allow you to specialize the iteration in a for:

• The built-in range function returns a list of successively higher integers, which can be used as indexes in a for.^[3]

  ^[3] Python also provides a built-in called xrange that generates indexes one at a time instead of storing all of them in a list at once like range does. There's no speed advantage to xrange, but it's useful as a space optimization if you have to generate a huge number of values.

• The built-in zip function returns a list a parallel-item tuples, which can be used to traverse multiple sequences in a for.

Let's look at each of these built-ins in turn.

10.4.1 Counter Loops: range

The range function is really independent of for loops; although it's used most often to generate indexes in a for, you can use it anywhere you need a list of integers:

>>> range(5), range(2, 5), range(0, 10, 2)
([0, 1, 2, 3, 4], [2, 3, 4], [0, 2, 4, 6, 8])

With one argument, range generates a list with integers from zero up to but not including the argument's value. If you pass in two arguments, the first is taken as the lower bound. An optional third argument can give a step; if used, Python adds the step to each successive integer in the result (steps default to one). Ranges can also be nonpositive, and nonascending, if you want them to be:

>>> range(-5, 5)
[-5, -4, -3, -2, -1, 0, 1, 2, 3, 4]

>>> range(5, -5, -1)
[5, 4, 3, 2, 1, 0, -1, -2, -3, -4]

Although such range results may be useful all by themselves, they tend to come in most handy within for loops. For one thing, they provide a simple way to repeat an action a specific number of times. To print three lines, for example, use a range to generate the appropriate number of integers:

>>> for i in range(3):
...     print i, 'Pythons'
...
0 Pythons
1 Pythons
2 Pythons

range is also commonly used to iterate over a sequence indirectly. The easiest and fastest way to step through a sequence exhaustively is always with a simple for; Python handles most of the details for you:

>>> X = 'spam'
>>> for item in X: print item,            # Simple iteration
...
s p a m

Notice the trailing comma on the print statement here, to suppress the default line feed (each print keeps adding to the current output line). Internally, the for handles the details of the iteration automatically. If you really need to take over the indexing logic explicitly, you can do it with a while loop:

>>> i = 0
>>> while i < len(X):                     # while loop iteration
...     print X[i],; i += 1
...
s p a m

You can also do manual indexing with a for, if you use range to generate a list of indexes to iterate through:

>>> X
'spam'
>>> len(X)                                # Length of string
4
>>> range(len(X))                         # All legal offsets into X
[0, 1, 2, 3]
>>>
>>> for i in range(len(X)): print X[i],   # Manual for indexing
...
s p a m

The example here is stepping over a list of offsets into X, not the actual items of X; we need to index back into X within the loop to fetch each item.

10.4.2 Nonexhaustive Traversals: range

The last example of the prior section works, but it probably runs more slowly than it has to. Unless you have a special indexing requirement, you're always better off using the simple for loop form in Python—use for instead of while whenever possible, and don't resort to range calls in for loops except as a last resort.

However, the same coding pattern used in that prior example also allows us to do more specialized sorts of traversals:

>>> S = 'abcdefghijk'
>>> range(0, len(S), 2)
[0, 2, 4, 6, 8, 10]

>>> for i in range(0, len(S), 2): print S[i],
...
a c e g i k

Here, we visit every second item in string S, by stepping over the generated range list. To visit every third item, change the third range argument to be 3, and so on. In effect, range used this way lets you skip items in loops, while still retaining the simplicity of the for. See also Python 2.3's new optional third slice limit, in Section 5.2.2 in Chapter 5. In 2.3, a similar effect may be achieved with:

for x in S[::2]: print x

10.4.3 Changing Lists: range

Another common place you may use range and for combined is in loops that change a list as it is being traversed. The following example needs an index to be able to assign an updated value to each position as we go:

>>> L = [1, 2, 3, 4, 5]
>>>
>>> for i in range(len(L)):          # Add one to each item in L
...     L[i] += 1                    # Or L[i] = L[i] + 1
...
>>> L
[2, 3, 4, 5, 6]

There is no way to do the same with a simple for x in L: style loop here, because such a loop iterates through actual items, not list positions. The equivalent while requires a bit more work on our part:^[4]

^[4] A list comprehension expression of the form [x+1 for x in L] would do similar work here as well, albeit without changing the original list in-place (we could assign the expression's new list object result back to L, but this would not update any other references to the original list). See Chapter 14 for more on list comprehensions.

>>> i = 0
>>> while i < len(L):
...     L[i] += 1
...     i += 1
...
>>> L
[3, 4, 5, 6, 7]

10.4.4 Parallel Traversals: zip and map

The range trick traverses sequences with for in nonexhaustive fashion. The built-in zip function allows us to use for loops to visit multiple sequences in parallel. In basic operation, zip takes one or more sequences, and returns a list of tuples that pair up parallel items taken from its arguments. For example, suppose we're working with two lists:

>>> L1 = [1,2,3,4]
>>> L2 = [5,6,7,8]

To combine the items in these lists, we can use zip:

>>> zip(L1,L2)
[(1, 5), (2, 6), (3, 7), (4, 8)]

Such a result may be useful in other contexts. When wedded with the for loop, though, it supports parallel iterations:

>>> for (x,y) in zip(L1, L2):
...     print x, y, '--', x+y
...
1 5 -- 6
2 6 -- 8
3 7 -- 10
4 8 -- 12

Here, we step over the result of the zip call—the pairs of items pulled from the two lists. This for loop uses tuple assignment again to unpack each tuple in the zip result (the first time through, it's as though we run (x,y)=(1,5)). The net effect is that we scan both L1 and L2 in our loop. We could achieve a similar effect with a while loop that handles indexing manually, but it would be more to type, and may be slower than the for/zip approach.

The zip function is more general than this example suggests. For instance, it accepts any type of sequence, and more than two arguments:

>>> T1, T2, T3 = (1,2,3), (4,5,6), (7,8,9)
>>> T3
(7, 8, 9)
>>> zip(T1,T2,T3)
[(1, 4, 7), (2, 5, 8), (3, 6, 9)]

zip truncates result tuples at the length of the shortest sequence, when argument lengths differ:

>>> S1 = 'abc'
>>> S2 = 'xyz123'
>>>
>>> zip(S1, S2)
[('a', 'x'), ('b', 'y'), ('c', 'z')]

The related, and older, built-in map function pairs items from sequences in a similar fashion, but pads shorter sequences with None if argument lengths differ:

>>> map(None, S1, S2)
[('a', 'x'), ('b', 'y'), ('c', 'z'), (None, '1'), (None, '2'), (None,'3')]

The example is actually using a degenerate form of the map built-in. Normally, map takes a function, and one or more sequence arguments, and collects the results of calling the function with parallel items taken from the sequences. When the function argument is None (as here), it simply pairs items like zip. map and similar function-based tools are covered in Chapter 11.

10.4.5 Dictionary Construction with zip

Dictionaries can always be created by coding a dictionary literal, or assigning to keys over time:

>>> D1 = {'spam':1, 'eggs':3, 'toast':5}
>>> D1
{'toast': 5, 'eggs': 3, 'spam': 1}

>>> D1 = {  }
>>> D1['spam']  = 1
>>> D1['eggs']  = 3
>>> D1['toast'] = 5

What to do, though, if your program obtains dictionary keys and values in lists at runtime, after you've coded your script?

>>> keys = ['spam', 'eggs', 'toast']
>>> vals = [1, 3, 5]

One solution to go from the lists to a dictionary is to zip the lists and step through them in parallel with a for loop:

>>> zip(keys, vals)
[('spam', 1), ('eggs', 3), ('toast', 5)]

>>> D2 = {  }
>>> for (k, v) in zip(keys, vals): D2[k] = v
...
>>> D2
{'toast': 5, 'eggs': 3, 'spam': 1}

It turns out, though, that you can skip the for loop altogether, and simply pass the zipped keys/values lists to the built-in dict constructor call in Python 2.2:

>>> keys = ['spam', 'eggs', 'toast']
>>> vals = [1, 3, 5]

>>> D3 = dict(zip(keys, vals))
>>> D3
{'toast': 5, 'eggs': 3, 'spam': 1}

The built-in name dict is really a type name in Python; calling it is something like a list-to-dictionary conversion, but really is an object construction request (more about type names in Chapter 23). Also, in Chapter 14, we'll meet a related but richer concept, the list comprehension, which builds lists in a single expression.