When measurement and test equipment are sent out for calibration most customers of calibration want a statement of compliance.

What is a Statement of Compliance?

Simply put, Meets Specifications or Out of Tolerance. In addition, most consumers want the calibration provider to make adjustments to bring the equipment back into tolerance. Another way of saying this is As Found and As Left data.

How do you interpret a statement of compliance?

ILAC provides Guidelines on Decision Rules and Statements of Conformity with ILAC-G8:09/2019.

The ILAC document provides the following:

Statements of Conformity

What are the implications for a Pass/Fail or out of tolerance statement?

In the white paper How to Maintain Your Confidence (in a World of Declining Test Uncertainty Ratios)

The following statement is made:

"Most calibration labs face the difficulty of having calibration standards which will not meet the desired or required TUR for some of the workload. The metrologist must choose to lower the level of confidence in the measurement, invest in more precise standards, or undergo an analysis of the uncertainties and document the deviations from the required TUR."

For more information on TUR you should read TUR:What is It?

Often a specification of a 4:1 TUR is given on RFQ's. There are several problems with this approach. The obvious is a 4:1 TUR is not always achievable and a 4:1 TUR does NOT eliminate all risk in decision making. We recommend you read WHY A 4:1 T.U.R. IS NOT ENOUGH: THE IMPORTANCE OF ANALYZING THE PROBABILITY OF FALSE ACCEPT RISK   for a thorough explanation of the risks involved with this approach.

One method that is used to address risk is called Guard Banding.

Guard Banding sets an Upper and Lower acceptance limit and if the measurement falls within the limits it would pass. The upper guard band would be slightly below the specification limit and the lower guard band would be slightly above the lower specification limit. An implication of Guard Banding is the probability of a measurement being outside of tolerance could be as high as 50% depending on the measurement uncertainty if the guard band is exactly the specification limits and it must be determined how measurement uncertainty will be accounted.

Source: Metrology Journal

Another common use of Guard Banding is to use a multiplier of measurement uncertainty to determine the band. This approach addresses the probability of a false accept or false reject when making a statement of conformity. This approach does not eliminate the risk of a False Accept or False Reject.

All measurements have an associated risk of calling something good when it is bad, Meets Specifications when it does not, and calling something bad when it is good, Out Of Tolerance when it is not.

In the paper, What TUR Do You Really Need? Putting Statistical Theory into Practice by Warren Wong Fluke Corporation there are some observations in regards to guard banding

"The results of this experiment are shown in Table 3, which shows the number of occurrences a UUT was found to be out-of-tolerance (OOT). (For the purposes of this study, a point not in-tolerance is called out-of-tolerance. This paper does not use the “indeterminate” condition defined in the Agy/Wong paper.) In every case but one, the ISO and Deaver method agreed when a measurement was OOT. In six of the ten cases, guardbanding declared the UUT to be OOT when no guardbanding would have found an in-tolerance condition. In four cases, each method gave the same results."

A simplistic way to look at this is the use of Guard Banding creates the tolerance specifications and risk you are willing to take with calibration measurements.

What Is Measurement Risk?

Quality Digest offered some insights on What is Measurement Risk?

"All measurements have a percentage likelihood of calling something good when it is bad, and something bad when it is good. You might be familiar with the terms “consumer’s risk” and “producer’s risk.” Consumer’s risk refers to the possibility of a problem occurring in a consumer-oriented product; a product that doesn’t meet quality standards passes undetected through a manufacturer’s quality control system and enters the consumer market."

Image Source: Quality Digest

 The article states that not using the laboratory with the right capability to meet your requirements is like using a ruler to calibrate a gauge block.

As a consumer of calibration services you need to evaluate calibration services based on the best measurement capabilities listed on the scope of accredited services and determine if the calibration and measurement capabilities are adequate for your use case for each piece of measurement and test equipment.

It should be evident by now that a calibration laboratory should be evaluated on the merits for each particular line item of services offered. Simply asking for "ISO17025" is woefully inadequate,

The end user of calibration services should determine the acceptable tolerance based on a statistical method that suits their operational requirements and evaluate calibration services based on their needs.

Simple Acceptance: a decision rule in which the acceptance limits is the sames as the tolerance limit

Mitutoyo has a clear definition: The rule is as simple as its name – measured values inside the tolerance limits are considered acceptable (simple acceptance) and those outside the tolerance limits are considered unacceptable (simple rejection).

Image Source: OIML

Image Source: OIML

With all these possible choices how is risk addressed?

Let's start with a definition of TUR- Test Uncertainty Ratio

TUR= UUT tolerance   / Std uncertainty

UUT=Unit Under Test

Std= Standard used for calibration

Z540 uses a more explicit definition of TUR.

TUR = Upper - Lower  / 2U

Test Uncertainty Ratio: The ratio of the span of the tolerance of a measurement quantity subject to calibration, to twice the 95% expanded uncertainty of the measurement process used for calibration.

Z540.3 requires that the unconditional false accept risk accompanying all calibrations be no greater than 2%.

ASME B89.1.13-2013, states that when assessing conformity with specification, a simple acceptance decision rule with a TUR ≥ 4 shall be used.

As stated above, a TUR > 4 is not always possible on every calibration and it does not eliminate all risk of a false accept or false reject.

As the end user of calibration, you will need to determine how you will address this issue. This conversation is best had before you send in items for calibration. Simply stating a TUR > 4 on your Purchase Order will just have your items on hold until it is resolved.

This also speaks to your RFQ for calibration services. It is still a common practice to simply ask for a "sticker" or "certificate" without specifying a tolerance or method used. It is also unreasonable to assume your calibration vendor knows what you need.  The consumer has information on the population of instruments types and use and this should inform the specifications and tolerance you request. "Manufacturers Specs" may be overkill or inadequate based on your actual use case.

Regardless of what method you use to manage your measurement risk, the final outcome in a statement of conformance should be customer driven not calibration provider driven.

Another way of stating this is if you receive a non-conformance at your next audit, the root cause is NOT the calibration provider if you did not specify exactly what you wanted.