Testability in Modern Product Engineering: Metrics, Verification, and Organizational Culture

By Jon M. Quigley

This post is in response to the link below.

https://www.linkedin.com/posts/lukastimm_nobody-needs-a-requirements-engineer-nobody-activity-7330245962235269125–3RC/?utm_source=share&utm_medium=member_android&rcm=ACoAAAJVUn4BzjTVfTnHgKx3zQWTPI5tYcYTiCM

The Power of Testability in Full-Stack Product Engineering

As Lukas Timm highlighted, the evolving landscape of automotive software development calls for a shift from siloed roles to empowered product engineers. Yet, in the drive for speed and ownership, the principle of testability must remain central to ensure quality, safety, and customer trust.

There are some benefits to having some of these silos. Silos provide a space for focus on a specific domain. This allows the team members to learn specific applications and develop approaches, tools, and even techniques in that domain.

Why Testability Matters

Testability—the ease with which a product can be tested—anchors Jon M. Quigley’s approach to quality. It ensures that teams do not sacrifice the rigor to deliver reliable, safe, and compliant automotive systems2 as they accelerate development cycles. In a world where engineers own the process end-to-end, testability becomes both a design principle and a cultural imperative.

Our testing, verification, and validation dictionary answers some questions on the need for independence.

Metrics and Data Gathering: The Backbone of Quality

Metrics and data gathering are foundational to assessing and improving testability. Quigley emphasizes that teams cannot reliably measure progress or identify risks without objective data. By integrating metrics such as defect density, code coverage, and test execution rates, organizations gain actionable insights into both product and process quality.

Defect Density: Tracks the number of issues per module, highlighting areas needing attention.
Code Coverage: Ensures critical paths are exercised, reducing the risk of undetected faults.
Test Execution Rates: Reveal bottlenecks and opportunities for automation.

Collecting and analyzing these metrics transforms intuition into evidence, supporting continuous improvement and informed decision-making.

Trust but Verify: Building a Culture of Accountability

A core tenet of Quigley’s philosophy is “trust but verify.” While empowering engineers to own the product lifecycle is vital, independent verification remains non-negotiable. This approach ensures that even as teams move fast, they do not become blind to their assumptions or biases.

“Trust but verify” means that while we rely on our engineers’ expertise and ownership, we also implement checks—through peer reviews, automated testing, and independent audits—to catch what might otherwise be missed.

This mindset fosters a culture where accountability and transparency are valued, not as bureaucratic hurdles, but as enablers of quality and innovation.

Some of our older test and verification magazine writing.

The Importance of Independent Verification and Testing

From experience, it is difficult to see the mistakes. This is the reason for design reviews. I recall working with Fred back in the day, sending him my Software Design Documents (or SRS) for review, and when I got them back, it was marked up in red. I still send him stuff for review, as I would rather he find the errors before the work gets launched.

In the rush to unify roles and accelerate delivery, the importance of independent verification and testing cannot be overstated. Independent verification provides an objective lens, ensuring that requirements are met and safety-critical functions perform as intended. This is especially crucial in automotive domains, where compliance and customer safety are paramount. Testing does not begin when the product (hardware/software) is delivered to the testing department. Well before that, the testing-responsible personnel will develop fixtures and test methods, collect test data, and write test cases, to name a few.

Experience shows that independent testing:

Uncovers integration issues that may be invisible to developers.
Validates that requirements are interpreted and implemented correctly.
Builds trust with customers and regulators by providing evidence of due diligence.

Our more recent magazine articles on product testing and verification.

Organizational Culture: Enabling Testability

For testability to thrive, organizations must cultivate a culture that values openness, learning, and continuous feedback. This means breaking down silos, encouraging collaboration, and rewarding transparency over blame. Quality becomes a shared responsibility when teams are empowered to share data, question assumptions, and embrace verification.

Conclusion

As the automotive industry races toward a more flat, product-focused engineering, testability remains a key phrase—and a non-negotiable pillar of quality. By embracing metrics and data gathering, fostering a “trust but verify” culture, and upholding independent verification, organizations can achieve both speed and excellence in product development. We are not confident that going too far down this road will be effective. We may think we cut the time to market down by doing so, but that may not be true beyond a certain level of complexity and risk. As always, there is no single silver bullet. The approach is the one that best serves the company, and not just during launch, but also the cost of quality once deployed and the associated risk due to product failure.