14.1 What the Test Harness Does

Tests are usually invoked (either for the developer or the installer) using make test. The Makefile invokes the test harness, which eventually gets around to using the Test::Harness module to run the tests.

Each test lives in a separate .t file in the t directory at the top level of the distribution. Each test is invoked separately, so an exit or die terminates only that test file, not the whole testing process.

The test file communicates with the test harness through simple messages on standard output. The three most important messages are the test count, a success message, and a failure message.

An individual test file consists of one or more tests. These tests are numbered as small integers starting with one. The first thing a test file must announce to the test harness (on STDOUT) is the expected test number range, as a string 1..n. For example, if there are 17 tests, the first line of output should be:

1..17

followed by a newline. The test harness uses the upper number here to verify that the test file hasn't just terminated early. If the test file is testing optional things and has no testing to do for this particular invocation, the string 1..0 suffices.

After the header, individual successes and failures are indicated by messages of the form ok N and not ok N. For example, here's a test of basic arithmetic. First, print the header:

print "1..4\n"; # the header

Now test that 1 plus 2 is 3:

if (1 + 2 =  = 3) {
  print "ok 1\n"; # first test is OK
} else {
  print "not ok 1\n"; # first test failed
}

You can also print the not if the test failed.^[6]

^[6] On some platforms, this may fail unnecessarily. For maximum portability, print the entire string of ok N or not ok N in one print step.

Don't forget the space!

print "not " unless 2 * 4 =  = 8;
print "ok 2\n";

You could perhaps test that the results are close enough (important when dealing with floating-point values):

my $divide = 5 / 3;
print "not " if abs($divide - 1.666667) > 0.001; # too much error
print "ok 3\n";

Finally, you may want to deal with potential portability problems:

my $subtract = -3 + 3;
print +(($subtract eq "0" or $subtract eq "-0") ? "ok 4" : "not ok 4"), "\n";

As you can see, there are many styles for writing the tests. In ancient Perl development, you saw many examples of each style. Thanks to Michael Schwern and chromatic and the other Perl Testing Cabal members, you can now write these much more simply, using Test::Simple.