A year ago Diabetes Care published our research letter [1] addressing the performance of current blood glucose meters (BGMs) in the low-glucose range. The initiative to write the letter was taken after studies showed considerable differences in performance in the low-glucose range between current BGMs [2, 3].
A good performing BGM in the low-glucose range is critical from a safety point of view to help patients with diabetes manage episodes of hypoglycemia. From a scientific point of view, good performance is needed to be able to confide in the data generated with the device during a clinical trial. In light of the varying performance between BGMs in the low-glucose range, the U.S. Food and Drug Administration (FDA) for example does not accept blood glucose measurements from BGMs as evidence for the efficacy of a new anti-diabetes drug with regard to reduction in hypoglycemic events.
We are not the only ones addressing the issue of poorly performing BGMs: the Diabetes Technology Society (DTS) launched the Blood Glucose Monitor Surveillance Program in May, 2014 [4]. The program aims to ensure that BGMs keep meeting minimum accuracy requirements after devices have been cleared by regulatory agencies for commercialization.
In March 2015, DTS finalised its testing protocol for studies evaluating the accuracy of BGM systems available on the US market. Although studies have started since June, 2016, no results were presented so far. At the annual DTS meeting in November 2016, the program initiators explained setting up the test program at various independent test centers and laboratories had proven to be more complicated than anticipated and results should be available at a next meeting.
The need for an accredited surveillance program, also outside of the USA, was painfully illustrated by a row between a low-cost BG manufacturer and an independent test center for BGMs in Germany. Results from the performance evaluation of the BGM and test strips bought at a local supermarket discounter were banned on judge’s order from presentation at the German national diabetes conference. The performance of the system tested did not meet the internationally recognised accuracy standards (that already leave quite some room for error, see below) and the poor performance especially in the low-glucose range could present a real danger to patients dosing insulin. Only after pressure from the German Diabetes Association and local media, the supermarket discounter started a call-back action of the low-performance BGMs.
The International Organization for Standardization (ISO) 15197:2013 [6] states that BGMs are acceptably accurate if ≥95% of their measurement results are found within ±15 mg/dL or ±15% (whichever is larger) of reference results. The 2014 FDA draft guidance [7] for over-the-counter BGMs requires ≥95% of results within ±15% and ≥99% of results within ±20% across the whole glycemic range.
This means that if a patient’s true BG concentration is 60 mg/dL, any measurement between 45 to 75 mg/dL would be considered accurate according to the ISO limits and between 51 to 69 mg/dL according to the FDA limits. In other words, a BG meter can receive regulatory approval without demonstrating it can reliably differentiate between 50, 60 or 70 mg/dL.
Furthermore, according to the ISO limits BGMs would still pass the accuracy test even if 1 out of 4 (!) measurements in the glucose range below 80 mg/dL are out (i.e. the deviation from the reference result is more than 15 mg/dL) and if all other measurements in the euglycemic and hyperglycemic region are within the accuracy limits.
For BGMs to get accepted as tool to generate efficacy data such as hypoglycemia reductions in clinical trials, the performance in the low-glucose range therefore needs to be redefined and reinvestigated. Profil previously presented that systematic investigations with the clamp method can provide robust and valuable data of BGM performance in any glucose range of clinical interest [8].
We also discussed, in our letter, (i) the meaning of the system accuracy requirements as defined in international standards or national guidance documents and (ii) the room these requirements leave for erroneous measurements in the low-glucose range.
Manufacturers of CGM sensors also have acknowledged that the frequent calibration from a BGM measurement is a potential source of larger error and therefore a weak link in generating accurate sensor data. Factory-calibration of CGM sensors circumvents this issue and the first generation of sensors that do not require any user calibration are now becoming available
The launch of the factory-calibrated sensors seems to be well-timed: In 2013 we reported [9] that CGM sensors were far less accurate than BGMs, especially in the low-glucose range and that this poor performance led to a failure to detect more than half of the true hypoglycemic events in our clinical trial. Furthermore, more than half of the alarms that were sounded to warn patients for hypoglycemia were false. A little over 2 years onwards, a recent publication by Peyser et al. [10] shows that a new-generation CGM sensor produces data in the low-glucose range that is on par with most BGMs: 90% of CGM values were within 15 mg/dL of the reference result with a mean absolute deviation of 6 mg/dL. The sensor investigated still required frequent calibration with a BGM, i.e. a superior sensor performance could not yet be expected, but the level of accuracy achieved was sufficient to detect 95% of all true hypoglycemic events in real-time.
Although CGM systems still face a similar challenge as BGMs to prove even more reliable and more accurate in the low glucose range, their larger data quantity and expected improvements in data quality when user and calibration errors are omitted are steps in the right direction to help these systems to become a useful and reliable tool for the assessment of hypoglycemia in future clinical trials.
Flash Glucose Monitoring is a hybrid form of blood glucose monitoring producing background continuous glucose monitoring data that becomes accessible every time a patient performs a manual sensor scan. The FGM system is the first generation of factory-calibrated sensors that do not require any user calibration and it is available in Europe since 2014, although still pending approval in the US. Patients welcomed the innovation as the system doesn’t involve finger prick calibration and the sensor can be used for up to 14 days.
Recently published randomised controlled studies [11] have shown that patients check their BG value about 15 to 20 times per day using FGM vs. about 5 times per day with a finger prick BGM. The increased number of checks and amount of data available may have contributed to the observed reduction of hypoglycemia and hyperglycemia in patients with type 1 diabetes in these studies.
Read about Error Grid Analysis, the widely accepted tool for clinical accuracy analysis. Else, learn more about implantable CGMs and watch our video on the topic; click here.