Steve September 16, 2020
teach-python-with-jupyter-notebooks

Some things about the Ruby community have always impressed me. Two examples are the commitment to testing and the emphasis on making it easy to get started. The best example of both is Ruby Koans, where you learn Ruby by fixing tests.

With the amazing tools we have for Python, we should be able to do something even better. We can. Using Jupyter Notebook, PyHamcrest, and just a little bit of duct tape-like code, we can make a tutorial that includes teaching, code that works, and code that needs fixing.

First, some duct tape. Usually, you do your tests using some nice command-line test runner, like pytest or virtue. Usually, you do not even run it directly. You use a tool like tox or nox to run it. However, for Jupyter, you need to write a little harness that can run the tests directly in the cells.

Luckily, the harness is short, if not simple:

import unittest

def run_test(klass):

    suite = unittest.TestLoader().loadTestsFromTestCase(klass)

    unittest.TextTestRunner(verbosity=2).run(suite)

    return klass

Now that the harness is done, it’s time for the first exercise.

In teaching, it is always a good idea to start small with an easy exercise to build confidence.

So why not fix a really simple test?

@run_test

class TestNumbers(unittest.TestCase):

   

    def test_equality(self):

        expected_value = 3 # Only change this line

        self.assertEqual(1+1, expected_value)

    test_equality (__main__.TestNumbers) ... FAIL

   

    ======================================================================

    FAIL: test_equality (__main__.TestNumbers)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 6, in test_equality

        self.assertEqual(1+1, expected_value)

    AssertionError: 2 != 3

   

    ----------------------------------------------------------------------

    Ran 1 test in 0.002s

   

    FAILED (failures=1)

Only change this line is a useful marker for students. It shows exactly what needs to be changed. Otherwise, students could fix the test by changing the first line to return.

In this case, the fix is easy:

@run_test

class TestNumbers(unittest.TestCase):

   

    def test_equality(self):

        expected_value = 2 # Fixed this line

        self.assertEqual(1+1, expected_value)

    test_equality (__main__.TestNumbers) ... ok

   

    ----------------------------------------------------------------------

    Ran 1 test in 0.002s

   

    OK

Quickly, however, the unittest library’s native assertions will prove lacking. In pytest, this is fixed with rewriting the bytecode in assert to have magical properties and all kinds of heuristics. This would not work easily in a Jupyter notebook. Time to dig out a good assertion library: PyHamcrest:

from hamcrest import *

@run_test

class TestList(unittest.TestCase):

   

    def test_equality(self):

        things = [1,

                  5, # Only change this line

                  3]

        assert_that(things, has_items(1, 2, 3))

    test_equality (__main__.TestList) ... FAIL

   

    ======================================================================

    FAIL: test_equality (__main__.TestList)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 8, in test_equality

        assert_that(things, has_items(1, 2, 3))

    AssertionError:

    Expected: (a sequence containing <1> and a sequence containing <2> and a sequence containing <3>)

         but: a sequence containing <2> was <[1, 5, 3]>

   

   

    ----------------------------------------------------------------------

    Ran 1 test in 0.004s

   

    FAILED (failures=1)

PyHamcrest is not just good at flexible assertions; it is also good at clear error messages. Because of that, the problem is plain to see: [1, 5, 3] does not contain 2, and it looks ugly besides:

@run_test

class TestList(unittest.TestCase):

   

    def test_equality(self):

        things = [1,

                  2, # Fixed this line

                  3]

        assert_that(things, has_items(1, 2, 3))

    test_equality (__main__.TestList) ... ok

   

    ----------------------------------------------------------------------

    Ran 1 test in 0.001s

   

    OK

With Jupyter, PyHamcrest, and a little duct tape of a testing harness, you can teach any Python topic that is amenable to unit testing.

For example, the following can help show the differences between the different ways Python can strip whitespace from a string:

source_string = "  hello world  "

@run_test

class TestList(unittest.TestCase):

   

    # This one is a freebie: it already works!

    def test_complete_strip(self):

        result = source_string.strip()

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world")))

    def test_start_strip(self):

        result = source_string # Only change this line

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world  ")))

    def test_end_strip(self):

        result = source_string # Only change this line

        assert_that(result,

                   all_of(starts_with("  hello"), ends_with("world")))

    test_complete_strip (__main__.TestList) ... ok

    test_end_strip (__main__.TestList) ... FAIL

    test_start_strip (__main__.TestList) ... FAIL

   

    ======================================================================

    FAIL: test_end_strip (__main__.TestList)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 19, in test_end_strip

        assert_that(result,

    AssertionError:

    Expected: (a string starting with '  hello' and a string ending with 'world')

         but: a string ending with 'world' was '  hello world  '

   

   

    ======================================================================

    FAIL: test_start_strip (__main__.TestList)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 14, in test_start_strip

        assert_that(result,

    AssertionError:

    Expected: (a string starting with 'hello' and a string ending with 'world  ')

         but: a string starting with 'hello' was '  hello world  '

   

   

    ----------------------------------------------------------------------

    Ran 3 tests in 0.006s

   

    FAILED (failures=2)

Ideally, students would realize that the methods .lstrip() and .rstrip() will do what they need. But if they do not and instead try to use .strip() everywhere:

source_string = "  hello world  "

@run_test

class TestList(unittest.TestCase):

   

    # This one is a freebie: it already works!

    def test_complete_strip(self):

        result = source_string.strip()

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world")))

    def test_start_strip(self):

        result = source_string.strip() # Changed this line

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world  ")))

    def test_end_strip(self):

        result = source_string.strip() # Changed this line

        assert_that(result,

                   all_of(starts_with("  hello"), ends_with("world")))

    test_complete_strip (__main__.TestList) ... ok

    test_end_strip (__main__.TestList) ... FAIL

    test_start_strip (__main__.TestList) ... FAIL

   

    ======================================================================

    FAIL: test_end_strip (__main__.TestList)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 19, in test_end_strip

        assert_that(result,

    AssertionError:

    Expected: (a string starting with '  hello' and a string ending with 'world')

         but: a string starting with '  hello' was 'hello world'

   

   

    ======================================================================

    FAIL: test_start_strip (__main__.TestList)

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "", line 14, in test_start_strip

        assert_that(result,

    AssertionError:

    Expected: (a string starting with 'hello' and a string ending with 'world  ')

         but: a string ending with 'world  ' was 'hello world'

   

   

    ----------------------------------------------------------------------

    Ran 3 tests in 0.007s

   

    FAILED (failures=2)

They would get a different error message that shows too much space has been stripped:

source_string = "  hello world  "

@run_test

class TestList(unittest.TestCase):

   

    # This one is a freebie: it already works!

    def test_complete_strip(self):

        result = source_string.strip()

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world")))

    def test_start_strip(self):

        result = source_string.lstrip() # Fixed this line

        assert_that(result,

                   all_of(starts_with("hello"), ends_with("world  ")))

    def test_end_strip(self):

        result = source_string.rstrip() # Fixed this line

        assert_that(result,

                   all_of(starts_with("  hello"), ends_with("world")))

    test_complete_strip (__main__.TestList) ... ok

    test_end_strip (__main__.TestList) ... ok

    test_start_strip (__main__.TestList) ... ok

   

    ----------------------------------------------------------------------

    Ran 3 tests in 0.005s

   

    OK

In a more realistic tutorial, there would be more examples and more explanations. This technique using a notebook with some examples that work and some that need fixing can work for real-time teaching, a video-based class, or even, with a lot more prose, a tutorial the student can complete on their own.

Now go out there and share your knowledge!

Read More