Most of us know that the “M” in the Six Sigma DMAIC acronym refers directly to the measurement and indirectly to the measurement system. Sometimes, it almost seems as if we have faith in measurement because it seems to be an objective activity as opposed to one based on opinion. However, measurement systems can produce their own distortions. For example, what does an IQ test measure? The best answer I have been able to come up with is that it measures your ability to take an IQ test. Does it measure the mythical intelligence value of “g” or does it measure largely nothing?
Furthermore, we might question the probability distribution function which, in this case, is nearly always a normal distribution. Is the PDF an artifact of the measurement system or does it truly represent what is going on in intelligence metrics? We already know that most continuous distribution functions are actually estimates for discrete distributions.
The point here is that it is certainly wise to avoid blithely assuming that we know what we are talking about when we measure a variable, particularly when that variable is not well understood (“g”). Positing hypothetical explanations that cannot be falsified violates Popper’s criterion for the advance of science.
The name of the game is “tread carefully.”