purl, URI templates and generated tests

TLDR: Parameterised tests are a useful testing technique. Both Nose and py.test support them.

URI templates in purl

The newly released purl 0.8 (a URL library of mine) supports URI templates as per RFC 6570. These allow new URL instances to be created by passing bindings to a template instance.

>>> import purl
>>> tpl = purl.Template('http://www.google.com{path}')
>>> tpl.expand({'path': ['a', 'b', 'c']}).as_string()
'http://www.google.com/a/b/c'

Alternatively, you can expand template strings directly:

>>> purl.expand('{?list*}', {'list': ['a', 'b', 'c']})
'/a/b/c'

There’s a plethora of ways template URLs can be used - see the RFC for details.

Note, there's already a Python library that provides this functionality. I decided not to use that one with purl as I thought I could do better. Plus, it looked like fun and I wanted to learn more about Python's support for parameterised tests.

Parameterised tests

I’d like to draw your attention to parameterised, or generated, tests which were used to drive development of this feature.

Example

The RFC includes a range of example templates, bindings and expected outputs:

image

These are excellent material for driving a series of parametric tests since each example comprises the inputs and expected output for a test. The natural way to test such examples is using parameterised tests.

PHPUnit

PHP’s dominant testing libary, PHPUnit, support data providers that can used to solve this problem (in a slightly clunky way).

There’s a Python port of this functionality in django-oscar’s testing utilities (and many other places no doubt) however there’s better way to write parameterised tests in Python.

Nose

I used Nose’s test generators to drive TDD on purl.

These allow tests to be generated by yielding a tuple (test_fn, *test_args) for each dataset. Here, test_fn is a callable that takes arguments test_args and should raise an AssertionError if the test fails.

Consider the above snippet from section 3.2.2 of the RFC: tests for this section can be constructed as:

import purl
from nose.tools import eq_

level1_vars = {
    'var': 'value',
    'hello': 'Hello World!',
}

# Tuples of (template, bindings, expected URI)
test_data = [
    ('{var}', level1_vars, 'value'),
    ('{hello}', level1_vars, 'Hello%20World%21'),
]

def assert_expansion(template, fields, expected):
    eq_(purl.expand(template, fields), expected)

def test_expansion():
    for template, fields, expected in test_data:
        yield assert_expansion, template, fields, expected

which executes each example as a single test:

$ nosetests tests/expansion_tests.py
tests.expansion_tests.test_expansion('{var}', {'var': 'value', 'hello': 'Hello World!'}, 'value') ... ok
tests.expansion_tests.test_expansion('{hello}', {'var': 'value', 'hello': 'Hello World!'}, 'Hello%20World%21') ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK
I have verbose output turned on by default since my ~/nose.cfg file contains the line verbosity=2.

Here we construct a simple assertion function using the eq_ equality check from Nose’s test tools.

The default verbose output is a little too verbose for my tastes. It can be cleaned up by providing a description attribute on the yielded callable:

def test_expansion():
    for template, fields, expected in test_data:
        assert_expansion.description = "%s expands to %s" % (template, expected)
        yield assert_expansion, template, fields, expected

which looks like:

$ nosetests tests/expansion_tests.py
{var} expands to value ... ok
{hello} expands to Hello%20World%21 ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Much nicer.

Updating the description suffers from a known bug where failure reports always use the last description assigned to callable. There are a few work-arounds detailed in the bug comments but none play nicely with Nose's multiprocess plugin.

py.test

In researching this post, I discovered py.test supports a rich array of functionality for creating parameterised or generated tests.

Using py.test’s @pytest.mark.parameterize decorator, we can rewrite the above example as:

import pytest

@pytest.mark.parametrize(("template", "fields", "expected"), data)
def test_expand(template, fields, expected):
    assert expand(template, fields) ==  expected

with verbose output:

$ py.test -v tests/pytest_tests.py
platform darwin -- Python 2.7.2 -- pytest-2.3.5 -- /Users/dwinterbottom/.virtualenvs/purl/bin/python
collected 2 items

tests/pytest_tests.py:123: test_expand[{var}-fields0-value] PASSED
tests/pytest_tests.py:123: test_expand[{hello}-fields1-Hello%20World%21] PASSED

I have a feeling I will be switching to py.test shortly.

Summary

Test generators are a useful addition to your testing toolkit.

There’s a couple of things to be aware of when using Nose’s generated tests:

  • It’s not possible to run just one of the examples from the commandline.
  • It’s tempting to create the assertion function inline within the test_* function. However this doesn’t work if you run your tests across multiple processes using Nose’s --processes option.

Here’s a few examples of using this functionality:

Related articles:

----

Tagged with: python, testing
Filed in: projects
Revision history

Previous: A deferred logging file handler for Django
Next: Dumping and restoring a PostGIS database

Copyright © 2005-2017 David Winterbottom
Content licensed under CC BY-NC-SA 4.0.