Investigating the Partial Relationships Between Testability and the Dynamic Range-to-Domain Ratio

Abstract

The word ‘testability’ has been used variously in the software community to represent a number of different concepts such as how easy it is to test a program or how easy it is to achieve execution coverage of certain program components. Voas and colleagues have used the word to capture a slightly different notion, namely the ease with which faults, if present in a program, can be revealed by the testing process. The significance of this concept is twofold. First, if it is possible to measure or estimate testability, it can guide the tester in deciding where to focus the testing effort. Secondly, knowledge about what makes some programs more testable than others can guide the developer so that design-for-test features are built in to the software. The propagation, infection and execution (PIE) analysis technique has been proposed as a way of estimating the Voas notion of testability. Unfortunately, estimating testability via the PIE technique is a difficult and costly process. However, Voas has suggested a link with the metric, domain-to-range ratio (DRR). This paper reviews the various testability concepts and summarises the PIE technique. A prototype tool developed by the authors to automate part of the PIE analysis is described and a method is proposed for dynamically determining the inverse of the domain-to-range ratio. This inverse ratio can be considered more natural in some sense and the idea of calculating its value from program execution leads to the possibility of automating its determination. Some experiments have been performed to investigate empirically whether there is a partial link between testability and this dynamic range-to-domain ratio (DRDR). Statistical tests have shown that for some programs and computational functions there is a strong relationship, but for others the relationship is weak.