July 3, 2016

Designing for Testability

How do we ensure that software does what it is built for? That it contains no bugs? We write tests. But writing good tests can be a challenge. The approach of writing tests separately or after production code is mostly obsolete. Instead, we employ practices like TDD and BDD, which put testing at the beginning of the actual development process. One reason this works out so well is that it constrains us to writing testable software.

Testability is the quality of a software system that makes it easy to isolate a part of it and demonstrate the presence of a bug or design flaw. Importantly, testability enables actual software engineering -- as in applying empirical methods to steer development. Testing for defects is just one such method.

Defect testing is risk management. Tests cannot be used to prove conclusively that a system is without bugs. Not that there is such a system anyway. Tests can only increase our confidence, that no critical bugs make it to production. We invest in the maintenance of a test suite to find bugs early, when they are still cheap to fix. Anything that lowers the cost of testing allows us to move towards earlier detection. Improving testability does just that.

Test Automation Pyramid

So what are indicators that software is not sufficiently testable? Well, it is not testable enough when we regularly find ourselves asking "how do I write a unit test for that?" It is really not testable enough when, in reaction to that question, we feel the urge to skip the more efficient unit or component tests and jump straight to the top of the test automation pyramid. Complicated test setup and habitual use of powerful mocking tools are also smells that indicate lack of testability.

Since defect testing is about managing risk, it should focus on those parts where the risk is highest. That is where the cost of failing to detect bugs early is significant and where it is less likely that we detect them early by other methods (e.g. compilation errors, deployment failures). Testable code should make this easy.

Testable software design separates things that are hard or less useful to test (e.g. file I/O, remote communication, library calls) from things that are hard to get right (e.g. complex business logic). The difficulty here is that these categories are not mutually exclusive: multi-threading, for example, tends to be both hard to test and hard to get right. It makes sense to me, that the more these categories are separated by design, the easier it is to test the important stuff.

Tags: Testability, Testing