purl, URI templates and generated tests

TLDR: Parameterised tests are a useful testing technique. Both Nose and py.test support them.

URI templates in purl

The newly released purl 0.8 (a URL library of mine) supports URI templates as per RFC 6570. These allow new URL instances to be created by passing bindings to a template instance.

>>> import purl
>>> tpl = purl.Template('http://www.google.com{path}')
>>> tpl.expand({'path': ['a', 'b', 'c']}).as_string()
'http://www.google.com/a/b/c'

Alternatively, you can expand template strings directly:

>>> purl.expand('{?list*}', {'list': ['a', 'b', 'c']})
'/a/b/c'

There’s a plethora of ways template URLs can be used - see the RFC for details.

Parameterised tests

I’d like to draw your attention to parameterised, or generated, tests which were used to drive development of this feature.

Example

The RFC includes a range of example templates, bindings and expected outputs:

These are excellent material for driving a series of parametric tests since each example comprises the inputs and expected output for a test. The natural way to test such examples is using parameterised tests.

PHPUnit

PHP’s dominant testing library, PHPUnit, support data providers that can used to solve this problem (in a slightly clunky way).

There’s a Python port of this functionality in django-oscar’s testing utilities (and many other places no doubt) however there’s better way to write parameterised tests in Python.

Nose

I used Nose’s test generators to drive TDD on purl.

These allow tests to be generated by yielding a tuple (test_fn, *test_args) for each dataset. Here, test_fn is a callable that takes arguments test_args and should raise an AssertionError if the test fails.

Consider the above snippet from section 3.2.2 of the RFC: tests for this section can be constructed as:

import purl
from nose.tools import eq_

level1_vars = {
    'var': 'value',
    'hello': 'Hello World!',
}

# Tuples of (template, bindings, expected URI)
test_data = [
    ('{var}', level1_vars, 'value'),
    ('{hello}', level1_vars, 'Hello%20World%21'),
]

def assert_expansion(template, fields, expected):
    eq_(purl.expand(template, fields), expected)

def test_expansion():
    for template, fields, expected in test_data:
        yield assert_expansion, template, fields, expected

which executes each example as a single test:

$ nosetests tests/expansion_tests.py
tests.expansion_tests.test_expansion('{var}', {'var': 'value', 'hello': 'Hello World!'}, 'value') ... ok
tests.expansion_tests.test_expansion('{hello}', {'var': 'value', 'hello': 'Hello World!'}, 'Hello%20World%21') ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Here we construct a simple assertion function using the eq_ equality check from Nose’s test tools.

The default verbose output is a little too verbose for my tastes. It can be cleaned up by providing a description attribute on the yielded callable:

def test_expansion():
    for template, fields, expected in test_data:
        assert_expansion.description = "%s expands to %s" % (template, expected)
        yield assert_expansion, template, fields, expected

which looks like:

$ nosetests tests/expansion_tests.py
{var} expands to value ... ok
{hello} expands to Hello%20World%21 ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Much nicer.

py.test

In researching this post, I discovered py.test supports a rich array of functionality for creating parameterised or generated tests.

Using py.test’s @pytest.mark.parameterize decorator, we can rewrite the above example as:

import pytest

@pytest.mark.parametrize(("template", "fields", "expected"), data)
def test_expand(template, fields, expected):
    assert expand(template, fields) ==  expected

with verbose output:

$ py.test -v tests/pytest_tests.py
platform darwin -- Python 2.7.2 -- pytest-2.3.5 -- /Users/dwinterbottom/.virtualenvs/purl/bin/python
collected 2 items

tests/pytest_tests.py:123: test_expand[{var}-fields0-value] PASSED
tests/pytest_tests.py:123: test_expand[{hello}-fields1-Hello%20World%21] PASSED

I have a feeling I will be switching to py.test shortly.

Summary

Test generators are a useful addition to your testing toolkit.

There’s a couple of things to be aware of when using Nose’s generated tests:

• It’s not possible to run just one of the examples from the commandline.
• It’s tempting to create the assertion function inline within the test_* function. However this doesn’t work if you run your tests across multiple processes using Nose’s --processes option.

Here’s a few examples of using this functionality:

• The expanded version of the above example from the Purl library
• A set of tests from django-oscar for testing validation of the Luhn algorithm.

Related articles:

• Introduction to Functional Web Testing with Twill and Selenium - Using test generators to drive functional tests.
• Nosetests, Generators and Descriptions - A detailed description of the bug with using a custom description for Nose’s generated tests.