We spend much money and energy trying to avoid uncertainty: we specify the characteristics of materials and components, test them and certify the results, and insist on evidence that the management systems of suppliers are certified to international standards. All this to control risk and make us more certain that nothing will go wrong or break when it shouldn’t. Define a process, always do the same thing in the same way, and we’ll get the same assured outcome, we think.
That can work well with manufacturing, where the characteristics of the inputs used in the defined process can be controlled, but it doesn’t work when we can’t control the inputs and where what is, just is. Then, if we do exactly the same things to varied inputs, we may be assuring that we get invalid outputs.
In testing, we can’t control what we have to test, or what the end user wants to do with the result. Instead, what we need is for the tester to be competent – which is to say that we want them to produce technically valid results that meet the needs of the user, and we want them to do that consistently across each test they perform.*
But really competent laboratories are never 100% certain about their measurement results, because that simply isn’t possible. Instead, for quantitative results, they evaluate the sources of uncertainty within their measurement system and the item being tested, and provide information on the overall uncertainty and its confidence level with the test result.
For all tests, including qualitative tests, the results derive from the test method used, so that a different method will give a different result. Recognizing that, competent testers are required to establish and state how closely they have been able to achieve the specified method, as the end user needs to decide for themselves whether the risk of applying the result to their own application is acceptable.
For competent testing and calibration laboratories, uncertainty is something to be quantified and stated, not ignored or hidden. If they only report the test measurement and specification limits but not the uncertainty of the measurement, how can it be known whether the material meets the specification? What if the uncertainty of the measurement overlaps the specification limit, so that the result is indeterminate?
Competent engineers need to know how reliable their data is; and competent testers and calibrators will tell them, whether they ask for it or not – how many people know what it is that they don’t know? If your competent measurement supplier isn’t admitting to being uncertain, they should be. If they won’t, you might want to know why not.
* The ISO standard describing what is necessary to achieve this is ISO/IEC 17025:2005, ‘General Requirements for the Competence of Testing and Calibration Laboratories’, and assurance that a laboratory is competent is given by independent assessment and accreditation by a body who is listed as a signatory to the Mutual Recognition Arrangement of ILAC (International Laboratory Accreditation Cooperation). For a signatory search, see www.ilac.org
Peter Kelley, a former senior metrologist at the National Weights & Measures Laboratory and training development manager at the United Kingdom Accreditation Service (UKAS), will give a speech entitled A measured approach to decision-making at The Institution of Engineering and Technology in London, UK, on Wednesday, February 3, 2016, at 7:15pm. Visit the IET website for more information.
Bio
Peter Kelley began employment in small manufacturing industry before joining the National Physical Laboratory in London, UK, working in engineering metrology. He became senior metrologist at the National Measurement Office (formerly National Weights & Measures Laboratory), then audit measurement officer at the National Measurement Accreditation Service, becoming an assessment manager and principal trainer and training development manager in its successor, the United Kingdom Accreditation Service (UKAS). He has assessed laboratory competence and delivered training and advisory services around the world. Following retirement in 2015, he remains active in consultancy and UKAS commercial training alongside other professional and personal interests.
