Sunday, October 16

Design by Contract in Python

In the light of recent lectures on Design by Contract in the university I decided see if there are any nice Python implementations.

To begin with, Design by Contract is a technique to increase reliability of programs, and in particular for reusable components. Invented by Bertrand Meyer more than ten years ago, it is a central notion in the programming language Eiffel that he created. There is a nice introduction to Design by Contract available. I'll try to summarize it very quickly.

Basically, in the context of software engineering a contract is a collection of obligations for a component. These obligations are divided into three major categories:

  • Class invariants are supposed to be (almost) always valid. For example, the attribute ISBN of a class Book should always be valid according to the ISBN checksum algorithm. A possible invariant for a class Window would be "not (window.maximized and window.minimized".
  • Preconditions are input conditions, they validate the arguments of a method or function call. For example, a precondition for the function sqrt(x) could be that its argument must be non-negative (assuming that it cannot deal with complex numbers). The caller of the function is responsible for supplying arguments that don't violate preconditions.
  • Postconditions are result obligations. They validate the result of a function/method and state changes of an object. An example of a postcondition for sqrt(): "abs(result * result - x) < 0.001". Postconditions can also access attributes and ensure that they have (or have not) been changed appropriately.

DBC might not look very pleasant for pythoneers at first glance, because it makes programs more rigid. Besides, unit and integration tests should catch all the problems, right? Well, not quite. I have experienced numerous bugs caused by violations of informally (if at all) described invariants. The problem with unit tests is that when you find a bug where one component (the "client") uses another component (the "server") incorrectly, you can fix the client and test for regressions in that client, but you cannot ensure that other clients have the same problem. Well, you can add assertion statements on the server, but I'm not particulary fond of assert statements as they clutter up the code and they are not suitable for checking a set of invariants in multiple locations. A nice thing about DBC is that, unlike static typing, which is an all-or-nothing affair, you can go only as far as you wish.

Preconditions in particular can be very beneficial to beginners. I have heard at least several people complain about dynamic typing in Python because they were frustrated by mysterious errors coming from the depths of a large framework which turned out to be a result of incorrect API usage. Comprehensive preconditions can eliminate such problems much better than any static typing system.

In Zope 3 applications that I worked with, a fair number of assumptions was supposed to hold at all times, they were mostly mentioned in interface docstrings, but only occasionally checked in actual code, and certainly not systematically. We've had a fair share of problems with objects referencing other objects that should have been deleted ("hanging in the air") and with processing inconsistent data structures. In relational databases, such requirements could be checked using constraints and triggers, but I am not aware of similar mechanisms for ZODB. In the end we bolted on a component that would swoop through all the objects in our application and check various things, but this component had to be invoked externally, using a cron job or manually pointing the browser at a particular URL. However, this was still far from perfect, because the component was separate, so requirements for components were not localized adjacent to the corresponding code. Using the Zope 3 component architecture would help with that, but it would increase the overhead for adding checks.

A possible complication with DBC in Python is the performance hit taken by all the extra checks. For compiled languages it's not nearly as bad as in Python where running unit-tests already takes minutes for larger applications. However, Python has always favoured convenience and doing the right thing over performance, so, given the benefits, this is not against the ideology at all. I do not like the current situation in zope.interface where if you want something checked against the interface, you have to run the check manually.

Of the Python implementations of DBC, I liked Contracts for Python most. In fact, its author Terence Way even proposed to include DBC in Python (see PEP-316), but the PEP was deferred. The general idea of the implementation is to declare the contract in docstrings, similar to doctest. This is much more lightweight and convenient than the other approaches which require to inherit from special classes (or set metaclasses) and/or define new special methods. Here is an example of a contract from the implementation's homepage:

def sort(a):
    """Sort a list *IN PLACE*.

        # must be a list
        isinstance(a, list)

        # all elements must be comparable with all other items
               lambda i: forall(range(len(a)),
                                lambda j: (a[i] < a[j]) ^ (a[i] >= a[j])))

        # length of array is unchanged
        len(a) == len(__old__.a)

        # all elements given are still in the array
        forall(__old__.a, lambda e: __old__.a.count(e) == a.count(e))

        # the array is sorted
        forall([a[i] >= a[i-1] for i in range(1, len(a))])

There is not much point in reiterating the concise documentation found in the package. You can find more examples on the "Contracts for Python" homepage.

It is unfortunate that the implementation is a bit stale, last touched more than two years ago as of today. I managed to find a trivial bug (patch) related to importing packages to be processed, but otherwise it seems to be still working fine. I also tried to make it work properly with Zope interfaces, so that implemented interfaces are treated as superclasses. Problem is, interfaces are not quite ordinary classes, so there were some complications. I might look further into it if anyone is interested. By the way, speaking about Zope interfaces, they already include support for invariants (see zope.interface.invariant), although I prefer the "Contracts for Python" way. Furthermore, preconditions and postconditions are not supported.


Holger Froebe said...

Hi Gintautas,

did you have a look at the
AspectOrientedProgramming (AOP) - Package for Python? It's much more current than Terence's DBC ...
and their approach
includes an "aspect" for DBC, but
is more general than pure DBC.
AOP looks very promising to me to help support separation of concerns.

The problem is: you have to explicit
"weave" the DBC-aspect into the Code.

But maybe (with some real programmer like you looking on the code) one could use this code to include it in Zope3 - I'd really love to see this feature.

I know that a lot of people in python-world tell me that AOP is
very easy and straightforward,
but I'm a scripter, not a
programmer, so my head explodes
while thinking about metaclasses.

caiyan said...

nike shoes
pandora charms
coach factory outlet
birkenstock shoes
coach factory outlet
ralph lauren
cheap nfl jerseys
louis vuitton outlet store
michael kors outlet clearance
ralph lauren outlet online