GNU Info

(python2.1-lib.info)Organizing test code

---

Next: Re-using old test code Prev: unittest Up: unittest

---

Enter node , (file) or (file)node

Organizing test code
--------------------

The basic building blocks of unit testing are "test cases" -- single
scenarios that must be set up and checked for correctness.  In PyUnit,
test cases are represented by instances of the `TestCase' class in the
`unittest' module. To make your own test cases you must write
subclasses of `TestCase', or use `FunctionTestCase'.

An instance of a `TestCase'-derived class is an object that can
completely run a single test method, together with optional set-up and
tidy-up code.

The testing code of a `TestCase' instance should be entirely self
contained, such that it can be run either in isolation or in arbitrary
combination with any number of other test cases.

The simplest test case subclass will simply override the `runTest()'
method in order to perform specific testing code:

     import unittest
     
     class DefaultWidgetSizeTestCase(unittest.TestCase):
         def runTest(self):
             widget = Widget("The widget")
             self.failUnless(widget.size() == (50,50), 'incorrect default size')

Note that in order to test something, we use the one of the `assert*()'
or `fail*()' methods provided by the `TestCase' base class.  If the
test fails when the test case runs, an exception will be raised, and
the testing framework will identify the test case as a "failure".
Other exceptions that do not arise from checks made through the
`assert*()' and `fail*()' methods are identified by the testing
framework as dfn{errors}.

The way to run a test case will be described later.  For now, note that
to construct an instance of such a test case, we call its constructor
without arguments:

     testCase = DefaultWidgetSizeTestCase()

Now, such test cases can be numerous, and their set-up can be
repetitive.  In the above case, constructing a "Widget" in each of 100
Widget test case subclasses would mean unsightly duplication.

Luckily, we can factor out such set-up code by implementing a method
called `setUp()', which the testing framework will automatically call
for us when we run the test:

     import unittest
     
     class SimpleWidgetTestCase(unittest.TestCase):
         def setUp(self):
             self.widget = Widget("The widget")
     
     class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
         def runTest(self):
             self.failUnless(self.widget.size() == (50,50),
                             'incorrect default size')
     
     class WidgetResizeTestCase(SimpleWidgetTestCase):
         def runTest(self):
             self.widget.resize(100,150)
             self.failUnless(self.widget.size() == (100,150),
                             'wrong size after resize')

If the `setUp()' method raises an exception while the test is running,
the framework will consider the test to have suffered an error, and the
`runTest()' method will not be executed.

Similarly, we can provide a `tearDown()' method that tidies up after
the `runTest()' method has been run:

     import unittest
     
     class SimpleWidgetTestCase(unittest.TestCase):
         def setUp(self):
             self.widget = Widget("The widget")
     
         def tearDown(self):
             self.widget.dispose()
             self.widget = None

If `setUp()' succeeded, the `tearDown()' method will be run regardless
of whether or not `runTest()' succeeded.

Such a working environment for the testing code is called a "fixture".

Often, many small test cases will use the same fixture.  In this case,
we would end up subclassing `SimpleWidgetTestCase' into many small
one-method classes such as `DefaultWidgetSizeTestCase'.  This is
time-consuming and discouraging, so in the same vein as JUnit, PyUnit
provides a simpler mechanism:

     import unittest
     
     class WidgetTestCase(unittest.TestCase):
         def setUp(self):
             self.widget = Widget("The widget")
     
         def tearDown(self):
             self.widget.dispose()
             self.widget = None
     
         def testDefaultSize(self):
             self.failUnless(self.widget.size() == (50,50),
                             'incorrect default size')
     
         def testResize(self):
             self.widget.resize(100,150)
             self.failUnless(self.widget.size() == (100,150),
                             'wrong size after resize')

Here we have not provided a `runTest()' method, but have instead
provided two different test methods.  Class instances will now each run
one of the `test*()'  methods, with `self.widget' created and destroyed
separately for each instance.  When creating an instance we must
specify the test method it is to run.  We do this by passing the method
name in the constructor:

     defaultSizeTestCase = WidgetTestCase("testDefaultSize")
     resizeTestCase = WidgetTestCase("testResize")

Test case instances are grouped together according to the features they
test.  PyUnit provides a mechanism for this: the `test suite',
represented by the class `TestSuite' in the `unittest' module:

     widgetTestSuite = unittest.TestSuite()
     widgetTestSuite.addTest(WidgetTestCase("testDefaultSize"))
     widgetTestSuite.addTest(WidgetTestCase("testResize"))

For the ease of running tests, as we will see later, it is a good idea
to provide in each test module a callable object that returns a
pre-built test suite:

     def suite():
         suite = unittest.TestSuite()
         suite.addTest(WidgetTestCase("testDefaultSize"))
         suite.addTest(WidgetTestCase("testResize"))
         return suite

or even:

     class WidgetTestSuite(unittest.TestSuite):
         def __init__(self):
             unittest.TestSuite.__init__(self,map(WidgetTestCase,
                                                   ("testDefaultSize",
                                                    "testResize")))

(The latter is admittedly not for the faint-hearted!)

Since it is a common pattern to create a `TestCase' subclass with many
similarly named test functions, there is a convenience function called
`makeSuite()' provided in the `unittest' module that constructs a test
suite that comprises all of the test cases in a test case class:

     suite = unittest.makeSuite(WidgetTestCase,'test')

Note that when using the `makeSuite()' function, the order in which the
various test cases will be run by the test suite is the order
determined by sorting the test function names using the `cmp()'
built-in function.

Often it is desirable to group suites of test cases together, so as to
run tests for the whole system at once.  This is easy, since
`TestSuite' instances can be added to a `TestSuite' just as `TestCase'
instances can be added to a `TestSuite':

     suite1 = module1.TheTestSuite()
     suite2 = module2.TheTestSuite()
     alltests = unittest.TestSuite((suite1, suite2))

You can place the definitions of test cases and test suites in the same
modules as the code they are to test (e.g. `widget.py'), but there are
several advantages to placing the test code in a separate module, such
as `widgettests.py':

   * The test module can be run standalone from the command line.

   * The test code can more easily be separated from shipped code.

   * There is less temptation to change test code to fit the code.  it
     tests without a good reason.

   * Test code should be modified much less frequently than the code it
     tests.

   * Tested code can be refactored more easily.

   * Tests for modules written in C must be in separate modules anyway,
     so why not be consistent?

   * If the testing strategy changes, there is no need to change the
     source code.