by: Shawn P. Quigley & Jon M Quigley

Measurements and Bias

Solely by the process of observing something we can alter the thing which is being observed. This is a known as the observer-expectancy effect.  This effect is born out of conscience and subconscious biases of the observer.  In the case of observing people, we have noted in earlier blog posts that the act of observation, taking an interest, may alter the outcome or performance as well (consider The Hawthorne study).  To make an effective measurement we must work to account for these impacts.  We also must know the goal we have set for collecting the information, that is the measurement is context based.  Having a specific goal, informs the type of data and methods of data collection.  Both of these are rife with opportunity for bias to creep into the measurements,  and delude our team.  This bias can creep in not only, what we identify as needed to measure, when the measurements are taken, but also in the interpretation of this information.  Our people who we are taking the measurements know what the measurement could mean for them; we again taint the validity of the information we are attempting to collect. Taking just these few issues into account it would seem that using functional measurements does not sound like a good idea.

Measurements Facilitate Understanding

Before we can take an effective measure of something to facilitate any form of change and/or improvement we must first full understand what the current processes, procedures, and personnel skills sets we are looking to improve.  This understanding where we are alone will likely require measurements.  This is the first step of any form of change management; knowing your actual starting point. Knowing from where one is starting is harder than most people would assume. Because preconceived notions of being better or just a lack of understanding of the work-around that are being employed to make the processes and procedures seem functional are rarely know by the people who desire a measurement to determine effective improvements.

It seems quite obvious as to why taking functional and operational measurements is necessary. It allows for trend forecasting, process improvement, and personal evaluation. When collecting information for trend analysis the process is mainly to record what is actually occurring. This differs from process improvement and personnel evaluation data collection in that some quantitative component must be involved to allow comparison.  It is this very quantitative component that drives the information collected and possibly the associated personnel behavior.

So Much Trouble – Measurements

This post would seem more to countermand any form of measuring: data collection, due to all the negatives brought forward. This is quite the contrary, knowing where a bias or gap in obtaining useful data for analysis is a must before starting on such an endeavor. Most processes or methods for gathering data for measuring something are costly and if employed improperly can lead to even more costly mistakes. This is the reason we start this segment on measures and analysis discussing these pitfalls.