Stochastic testing occurs when we allow a reasonably well-seasoned test engineer to go with their “gut” and feel their way about the product’s performance. During the development of numerous embedded automotive products, we have seen stochastic testing elicit roughly the same amount of test failures as combinatorial testing. We are not recommending that stochastic testing […]
We know that a very simple product or system can generate a vast number of potential test cases. With more complex systems, this number becomes astronomical. This is the result of a factorial calculation! One technique we use to get around this problem originates with designed experiments. Many designed experiments are based on orthogonal […]
We have discussed compliance testing earlier. This is known as testing to requirement. These requirements can be taken directly from a customer specification (when we have one) or derived internally from a requirements review or even both. Compliance testing is the primary method we use to ensure that we are meeting all specifications and […]
To perform embedded software testing, we recommend five phases of testing. These phases may take place concurrently and are as follows: Compliance testing Combinatorial testing Stochastic (or exploratory) testing Extreme testing Attack mode Each of these methods provides some view of the product the other method does not. Of course, testing the product is not […]
Requirements management and configuration management are required for anything that even closely resembles effective testing. Experience suggests failing in these two areas unnecessarily complicates the product verification activities, and we will show some of those traumas in the next few posts. An iterative and incremental product development process calls for reviews throughout the development process. […]
We have our requirements, our iterative packages and content defined, and now the developers are producing the iterations. During this time, the verification and test folks have been creating the test cases or details on how we intend to confirm the product performance. Our requirements provide the fodder for our testing. For each requirement there […]
Consider a rather large project that like so many projects had some difficulties. The project team had a major component (subsystem) delivered from a supplier. The supplier has one set of processes and the customer organization another. This supplier delivers multiple versions of this major subsystem. The customer integrates this subsystem into the larger system […]
Creating a separate software test group has pluses and minuses. At a minimum, we may have more to manage. Some of the minuses are a product of human nature. When we know our work will be inspected, we will often assume the inspector will catch issues and we pay less attention to the issue ourselves. […]
We have been on a bit a tear (or rant) about FMEAs. We suggest the FMEA documentation is part of the core of a design process. The ultimate approach we have seen is that of Michael Anleitner (The Power of Deduction: Failure Modes and Effects Analysis for Design, Quality Press, 2010), which uses functional analysis […]
There is only one way to describe this scenario and that is via a story. Consider the organization that is coming to the end of the project. The product is a complicated subassembly that goes into a larger system and has numerous interactions and incarnations of the design. They are late in the delivery of […]