Refinement Testing refers to the use of tests for Multiple-Failure isolation that only result in reduced ambiguity when they pass. During Multiple-Failure isolation, a refinement test is any test whose coverage contains some (but not all) of the functions or failure modes covered by a previously failed test, and which contains in its coverage at least one function or failure mode not covered by the previous test. Consider the following example:
In this sample, test T2 has failed and the input to function A is assumed to be good. The suspected set of functions initially consists of functions ABDE (the coverage of test T2). The other two tests (T1 and T3) are both considered refinement tests. Both tests cover some, but not all, of T2's coverage (T1 covers AB, whereas T3 covers A). Also, both tests contain in their coverage functions not covered by T2 (T1 covers C; T3 covers F). A multiple-failure diagnostic strategy would now attempt to reduce the set of suspected functions using T1 and T3 as refinement tests.
If the refinement test T1 was performed next and that test were to pass, the diagnostic engine would conclude that ABC (the coverage of test T1) are good and remove them from the suspect set (which would now only consist of functions DE, as depicted above). If test T1 were to fail, however, the suspect would not be reduced:
Because T1 is a refinement test, it does not result in reduced ambiguity when it fails. The suspect set remains the same (ABDE), although function C is identified as being a secondary suspect (since it may have caused the refinement test to have failed). The diagnostics will next attempt to reduce ambiguity using test T3:
If test T3 were to pass, functions AF would be proven good and the suspect set would be reduced to BDE (as depicted above). If T3 were to fail, however, it would once again result in no reduction of the suspect set:
Here, F has additionally been identified as a secondary suspect (since it could have caused T3 to have failed), but the isolated fault group is still ABDE.
Note: in this example, tests were chosen in an order that best depicts refinement testing. Actual diagnostics may apply multiple-failure inference in a manner that produces more optimal fault isolation.
