Software Development Practices: Testing for Behavior, Not Just Success

This is the fourth and final post in our software development practices series. In our most recent post we discussed how Acceptance Criteria could be used to encapsulate the details of the user experience that the system should provide. This week we'll talk about how developers can use tests to determine whether or not the system is satisfying the Acceptance Criteria.

An Overview of tests

There are several different types of tests developers can use to ensure the system is working properly. Unit tests, integration tests, and acceptance tests are the most common. Each of these types of tests exercise different aspects of the system:

• The purpose of unit tests is to ensure that individual units of code such as classes and functions perform properly.
• Integration tests ensure that code across individual “units” work properly together. Integration testing is necessary to test behavior that spans components or systems and are good to use for code relies on a database, web service, or any other external system.
• Acceptance testing ensures that the system meets its intended purpose.

I have recently become a fan of Behavior driven testing, which complements the development practices we’ve covered in this series so far. Before I get to that though, let me give you a little bit more background on test driven development to set the stage.

Test driven development

One pattern for creating tests as part of the development process is Test Driven Development (TDD). In TDD, the first step a developer takes when creating any new functionality is to write an automated test which will validate whether or not the functionality is working. As this is the first step, the test will always initially fail - the code which would make the test pass has not been written yet. Once the developer runs the test suite and verifies the test fails, they then write the code that will satisfy the test. Then the developer runs the tests again making sure all the tests pass. This is especially valuable as the code becomes more complex and the number of tests grows because, as all developers know first hand, any change to code might break existing functionality within the system. The process is repeated until all the new functionality has been added to the system.

One weakness of TDD is that it doesn't specify at what level any given test should be implemented. Tests can be at the high-level requirements, low-level function details or anywhere in between. As a developer, I could use test driven development techniques to only write unit tests and not integration or functional tests. Because of this it is possible to have tests that do not adequately cover the product owner's acceptance criteria. Another drawback is that tools traditionally used for unit or integration testing do not produce tests which are easily readable by product owners. Here’s an example of an automated acceptance test written using PHPUnit.

Without comments in the code it would be incredibly difficult to know what behavior is being tested and what the expected outcome is supposed to be. This example is actually one of a series of three tests within the AddressTest test group in the WorldCat Registry Demo Application.

Behavior driven development

As a result of these shortcomings, another testing methodology has gained favor in the last 10 years. Dan North and others developed Behavior Driven Development (BDD) as a way to frame communication between developers, testers, and product owners. It follows the same techniques and principals as TDD, but seeks to deal with its shortcomings.

To do this, BDD asserts that the desired system behavior is what is always being tested. Additionally, tests are written using natural language so that they are easily understood by everyone, including product owners. In fact, some of the most popular BDD tools use the same Given, When, Then syntax we discussed in relation to acceptance criteria. Because Behavior driven tests are focused on testing the desired system behavior, they can more easily be tied to User Stories and acceptance criteria. In fact, some BDD tests include the user story and acceptance criteria:

You can see from looking at this example that the tests are human readable and use a format similar to user stories and acceptance criteria. The test or “feature” in BDD language corresponds to a single story. The “scenarios” within the “feature” each correspond to an acceptance criteria which is being tested. In our sample WorldCat Registry Hours application there are three features, one for each of the user stories.

Testing tools

Depending on your programming language of choice, there are a number of different testing tools available. Often these testing tools are available as an additional library. For example, PHPUnit is a very popular unit testing library for PHP. BDD testing libraries also exist and come in two flavors.

The “specification” flavor uses functional specifications as the input format for test scenarios. Rspec and PHPSpec are examples of this. These specifications are more technical in nature and not necessarily as effective for facilitating communication between developers and product owners.

The “story” flavor uses the Gherkin language for writing tests. This is the flavor of BDD tool I prefer because it closely ties tests, stories and acceptance criteria. These BDD testing libraries typically have code which transforms the natural language story and scenario statements into actions that execute the tests. The standard actions that come with a library can be extended by developers to suit a specific project’s needs. One popular tool for Behavior Driven Development is called Cucumber. There is a PHP implementation of this called Behat. The WorldCat Registry Hours Demo Application uses a combination of PHPUnit for unit testing and Behat for acceptance testing.

This post concludes our series on software development practices. We hope you have found some useful information for bridging the communication gaps between product owners and developers to build more collaborative development processes. We're planning to follow this series up with a couple of webinars on these topics. Stay tuned!