Written By Mike Carnell and Lisa Dolf Herkenrath
A question was posted on the Internet asking what this joke meant. There are height strips posted on the door of most convenience stores to be used in case they are held up. As the perpetrator leaves the store they will pass the height strip, and the description that is provided to the police will have some level of accuracy beyond a person’s best guess. That accuracy is only useful if the height strip is accurately installed. If it was not placed correctly against the floor and a 6-foot person appears to be 6 foot 6 inches, the height estimate will be inaccurate, and the description will be incorrect.
The point is that just because a measurement device is used to make a measurement that does not necessarily make that measurement true. That same logic works on every measurement device/system. It is even worse when it is a device operated by a person, so you then have both the device error and the operator error. This is why Measurement System Analysis (MSA) is so critical for day-to-day operations and continuous improvement efforts.
Measurement System Analysis
This is also called MSA and was previously known as GR&R. GR&R is a truncated version of MSA which only addresses precision and not accuracy. It can be defined as:
An experimental and mathematical method of determining how much the variation within the measurement process contributes to overall process variability. There are five parameters to investigate in an MSA: bias, linearity, stability, repeatability, and reproducibility.
This is a pretty simplistic definition. There are other factors involved in MSA, but this a good working definition with which to begin.
It is important to understand that when we use the word “system” we are going beyond the evaluation of a gage. We are looking at the system that operates the gage including factors such as the person operating the gage, the parts being measured, and the fixtures holding the parts, etc. We are talking about the systemic error that is present in the data from the object being measured that can be attributed to the gage rather than the object.
Gage Defined
Again, simplistically, this is an instrument or device for measuring the magnitude, amount, or contents of something, typically with a visual display of such information. This simplistic definition creates difficulty in people recognizing that a gage does not have to be an instrument or device. In some instances, it may be a person.
To understand more clearly, if we take the definition of a gage as a verb rather than a noun we have to estimate or determine the magnitude, amount, or volume. If we understand the function, i.e. the verb or action, it becomes easier to understand that it does not need to be restricted to an instrument or device. People completing a visual inspection are also behaving as a gauge.
Introduction
All gages have error associated with every measurement or piece of data they produce. That is a fact. The intent of the MSA is to understand how large that error is and its effect upon whatever is being measured.
Types of Error
There are two very basic types of error. Logically enough they are called Type 1 and Type 2 errors. Type 1 is frequently referred to as a false positive. Type 2 is a false negative.
We are hearing this referred to frequently these days in terms of drug testing. A false positive means a person tested positive for a particular drug when in reality he/she did not have that drug in their system. That would be a Type 1 error. A false negative means the person tested did not show a particular drug in his/her system when in reality that drug was present.
Basically, it looks like this:
⦁ True Positive = Tested positive when in a person was positive
⦁ False Positive = Tested positive when a person was negative
⦁ True Negative = Tested negative when a person was negative
⦁ False Negative = Tested negative when a person was positive
Measurement error is not new. It has been well known for a long time, but it has not been and still is not understood well. When people discuss measurement error the typical response is “we calibrate our equipment.” Calibration, technically is referred to as bias, is only one of five characteristics of most measurement systems.
Accuracy and Precision
Measurement Systems Analysis addresses two major characteristics. One is accuracy, which is the ability to hit a specific target. Precision is the second characteristic and is the ability to get the same measurement multiple times and by multiple systems.
The above diagram represents the various combinations of accuracy and precision. Of course, it is important to be able to be both accurate and precise.
The elements of a measurement system analysis that needs to be addressed look like this:
Many diagrams represent MSA mixed with part variation. This representation is specific to the measurement system. We want to separate the variation attributed to the measurement system. That allows us to better understand the true process variation. There is no point in making this topic more complex by including part variation. The basic premise is this: If I had multiple parts, for example 10, that were exactly the same, if these parameters were not under control, it could easily appear as if every part were different due purely to the measurement system.
In the chart shown below the ideal situation would be to have the actual equal to the observed. That, of course would be a 45-degree line passing through the origin in a Cartesian coordinate system. As you can see, as gage error increases the process appears to be less capable when in fact it is a function of the gage not the process.
In the beginning of the movie The Big Short they use a Mark Twain quote “It ain't so much the things that people don't know that makes trouble in this world, as it is the things that people know that ain't so.” Why would that be part of a movie about a Subprime Mortgage Crisis? Consider investments were made based on a risk assessment. Companies assign a risk metric to investment using what should be an interval scale or at worst ordinal scale. Things like AAA, AA, A, etc. are meant to represent risk. The movie makes a major point that those ratings did not reflect reality. That is a measurement system failure. That measurement system failure cost was $1,488B. That is exactly what Mark Twain was trying to convey so long ago. The trouble begins with what people believe they know, and it is not always true.
In a world that spends a lot of time discussing Big Data, data analytics, and data driven decision-making, those entire processes are a function of the quality of the data with which you begin. The data you begin with must originate somewhere and that somewhere is almost always through a measurement system.
There was a time when any discussion about data analysis of any type would lead to someone throwing out the cliché “garbage in, garbage out” just so they could feel like they had contributed during the conversation. This cliché is actually true, but if you were to question the person presenting it with “How do I determine if the data is garbage” in general, (he/she is generally clueless) people who consider clichés as insight will not recognize MSA as a solution. The correct answer is MSA. Measurement systems are critical to good decision-making. The quality of the measurement system is the output of a MSA and is a direct measurement of the quality of the data with which you will begin your analysis.
Measurement Systems Analysis (MSA) is a critical component of data analysis and is critical to the integrity of the analysis. Despite having built-in types of error, we can assume a certain amount of accuracy and precision within all measurements if acquired correctly. MSA is essential not only for day-to-day operations but also for continuous improvement endeavors. According to the internet, the consensus is that Ron White is 6’2” regardless of what convenience store he is leaving. Any other measurement is a pure measurement error.