Compliance Testing
We have discussed compliance testing earlier. This is known as testing to requirement. These requirements can be taken directly from a customer specification (when we have one) or derived internally from a requirements review or even both. Compliance testing is the primary method we use to ensure that we are meeting all specifications and any contractual obligations. Compliance testing is therefore obviously an important part of the project management activities. In the automotive industry, for example, we will verify that all gauges are meeting pointer sweep requirements in our instrument cluster (where we find the gauges on our dashboards).
Compliance testing has the defect of only testing for specification requirements and may provide an inadequate measure of the true performance of the system. Compliance testing will not show us how atypical or non-specified interactions with the product will impact the performance or product longevity. Additionally, it can be difficult to record by using specifications all of the unwanted behaviors the product must not have. To do so we would amount to documenting all the things the product must do, and all of the things the product will not do. It is not very efficient, for example, to take time to identify all of the scenarios in which the product must not lock up. It should not lock up under any conditions. We expect our test engineers to be endeavoring to discern where the product fails rather than trying to prove that the product works! This idea leads us to our next phase of testing.