Assessing clinical and statistical agreement between an innovative point-of-care medical device and a conventional glucometer in measuring random blood glucose

Diabetes mellitus is rapidly emerging as a grave public health problem worldwide, more so for the low-income countries of the South-east Asia and Africa. Globally, type 2 diabetes mellitus (hereafter diabetes) is responsible for around 90% of all cases of diabetes. To put things in perspective, the global increase of more than 200 percent [1]. It is projected that there would be 47 million diabetic cases in Africa and another 153 million cases in South-east Asia in the year 2045, an expected relative increase of 143 percent and 74 percent, respectively, vis-à-vis the current prevalence [2].

diabetes prevalence among adults aged 20—79 years The prevalence of adult diabetes in India is has increased from 151 million in the year 2000 to as approximately 10.4 percent, and is rising. In fact, much as 463 million in 2019-a mammoth relative

                   

International Journal of Medical Science and Current Research | May-June 2020 | Vol 3 | Issue 3

 

India is home to the second largest number of adults living with diabetes globally. Paucity of reliable diagnostic services at the last-mile to screen the base of the pyramid population for diabetes remains a bane for resource-constrained countries such as India. Notably, evidence suggests that there are around 43.9 million people in India living with undiagnosed diabetes [1]. While mostly considered a morbidity affecting the urban affluent, diabetes is becoming increasingly prevalent among rural populations in India and is contributing to unabated health iniquity [3,4]. It may be reminded here that early diagnosis of diabetic cases through effective screening is of paramount significance to preclude development of deleterious and cost-intensive medical complications among these diabetic patients [5].

In this background, this study was undertaken with the primary objective to assess the clinical and statistical agreement between an innovative blood monitoring system, named AINA, and a conventional glucometer in measuring random blood glucose for screening patients for diabetes. The study was undertaken in the primary healthcare settings in the state of Rajasthan in India. AINA, apart from random blood glucose, measures haemoglobin, lipid profile, and glycated albumin using finger-stick capillary whole blood. Through these additional tests, AINA tends to position itself as a comprehensive diabetes prevention and care platform encompassing definitive diagnosis of diabetes using glycated albumin and diagnoses of comorbidities such as anemia and dyslipidemia which contribute to progression of diabetic complications among diabetics [6]. However, this study is limited to random blood glucose measurement as the same is used for screening patients for diabetes in primary healthcare settings as per the extant government of India guidelines [7]. A reliable and accurate measurement of random blood glucose using AINA could at least help to screen patients in far-flung areas in India for further clinical workup.

METHODS

A prospective study was conducted in January 2020 in primary healthcare settings in Rajasthan, India.

Patients aged above 30 years who visited the identified primary health centres and who were advised random blood glucose test by the attending medical doctor based on comprehensive history taking and thorough clinical examination were enrolled in the study. All the subjects underwent the test through both AINA and the conventional glucometer using capillary fingerstick whole blood samples by trained laboratory technicians. Age, sex, and random blood glucose levels (in mg/dl) were reported in prescribed formats (separate for AINA and the conventional glucometer) for each of the capillary fingerstick whole blood sample drawn from the subjects. All the subjects were given standard care and treatment for diabetes, as applicable, and all clinical decisions were based solely on the blood test reports of the conventional glucometer. Assuming the sample size to give a Cohen’s kappa value of 0.8 with the maximum acceptable 95% confidence interval width of 0.2 and the true proportion of positives to be 10% for diabetes, the required sample size came out to be 401. Alternatively, on an assumption of an anticipated Lin’s concordance correlation coefficient of 0.8 with the maximum acceptable 95% confidence interval width of 0.10, we arrived at an optimal sample size of around 200 subjects [8].

Klonoff’s surveillance error grid was employed to assign a level of associated clinical risk to random blood glucose values measured using AINA [9,10,11]. Deming regression model was used to perform method comparison and analysis. Cohen’s kappa coefficient (κ) was calculated to measure the degree of agreement on the presence of diabetes whereas Lin’s concordance correlation coefficient was calculated as an index of reliability to measure the degree of agreement on random blood glucose values. IBM SPSS Statistics version 24 was used for basic statistics, whereas Statistical MedCalc Version 18.10 was used for advanced statistics.

Written informed consent was obtained from all the subjects. The confidentiality and privacy of the subjects were maintained throughout the study. A personal identification number was assigned to each subject for identification and follow up. Entire data on the device were password-protected. The institutional ethical committee of the WISH Foundation approved the study.

RESULTS

During the study period, a total of 465 subjects (237 men [51%] and 228 women [49%]) underwent random blood glucose test using AINA and the conventional glucometer. The average age of the subjects was 46.6 years (95% confidence interval [CI]: 45.1 to 48.2 years). Considering a random blood glucose level of 200 milligrams per decilitre (mg/dL) or higher as suggestive of diabetes, 66 (14.2%) subjects were screened positive using the conventional glucometer.

Results of employing the Klonoff’s surveillance error grid on the random blood glucose levels measured using AINA and conventional glucometer are shown in table 1.

Table 1: Showing the results of employing Klonoff’s surveillance error grid on the random blood glucose levels measured using AINA and conventional glucometer

 

Pair Types
1. Total pairs included in SEG analysis 465
2. AINA<REF 238
3. AINA=REF 14
4. AINA>REF 213
Agreement
1. Bias 0.4%
2. MARD 5.9%
3. CV 7.5%
4. Lower 95% limit of agreement -14.3%
5. Upper 95% limit of agreement 15%
Pairs as per SEG Risk Category
1. Zero (no risk) 452 (97.2%)
2. One (slight, lower) 13 (2.8%)
3. Two to Seven (moderate to extreme risk) None
Overall compliance
1. Total compliant pairs 449 (96.6%)
2. Lower bound for acceptance 434 (93.3%)
3. Overall 96.6% > 93.3%; AINA meets BGM Surveillance

Study Accuracy

Standards

SEG: Klonoff’s Surveillance Error Grid; REF:

Reference/Conventional Glucometer; Bias: Mean Relative

Difference between AINA and REF; MARD: Mean Absolute

Relative Difference; CV: Standard Deviation of Relative

Difference between AINA and REF; Lower 95% Limit of

Agreement: Bias – 1.96 * CV; Upper 95% Limit of Agreement: Bias + 1.96 * CV; BGM: Blood Glucose Monitoring

 

Notably, out of a total of 465 pairs, 452 (97.2%) were in zero risk category and 13 (2.8%) were in risk category one (slight, lower). Overall, 449 pairs (96.6%) were fully compliant with the standards of the blood glucose monitoring system surveillance program.

Figure 1: Pictorially depicting the measured blood glucose levels (AINA) versus the reference blood glucose levels (conventional glucometer) across the five risk levels

 

Table 2 shows the method comparison using the Deming regression model. As is clear from the table, the 95% confidence interval for the Intercept contains the value 0 and the 95% confidence interval for the slope contains the value 1. Therefore, the two methods of measuring the blood glucose levels, i.e. AINA and the conventional glucometer, do not statistically differ from each other. There is neither any statistically significant systematic difference nor any proportional difference between the two methods in measuring blood glucose levels.

Cohen’s kappa coefficient (κ) calculated to measure the degree of agreement on the presence of diabetes in the study sample came out to be 0.9642 (standard error: 0.0178; 95% confidence interval: 0.9293 to 0.9991), whereas Lin’s concordance correlation coefficient calculated as an index of reliability to measure the degree of agreement on random blood glucose values came out to be 0.9886 (95% confidence interval: 0.9863 to 0.9905; Pearson correlation coefficient [precision]: 0.9886; bias correction factor Cb [accuracy]=1.0000).

Table 2: Showing the method comparison using the Deming regression model

 

Deming Regression Model 

(Method y: AINA; Method x: Conventional Glucometer)

Regression equation: y=0.9058+0.9937x

Method y: mean=125.98 mg/dl; coefficient of variation=41.19%

Method x: mean=126.10 mg/dl; coefficient of variation=40.89%

Pearson correlation coefficient: 0.9886 (95% Confidence Interval: 0.9864 to 0.9905)

Parameter Coefficient Standard Error 95% Confidence Interval
Intercept 0.9058 1.0106 -1.0801 to 2.8918
Slope 0.9937 0.007390 0.9792 to 1.0083

 

DISCUSSION

The Klonoff’s surveillance error grid is an enhanced modification of Clarke error grid and Parkes error grid [12,13,14]. It is a robust tool to assign a quantum of clinical risk associated with inaccuracies, if any, in measurements using different blood glucose monitoring systems [9]. The grid precisely and accurately quantifies the clinical risk and uses a colour-code to represent the same. This clinical risk assessment, over and above the statistical agreement, is of pivotal significance as even relatively small errors in blood glucose monitoring system may lead to catastrophic clinical results in management of diabetic patients and also in clinical management aimed at precluding development of medical complications in these patients [15]. The level of clinical risk associated with each pair of AINA and conventional glucometer results is represented by colour coding on the grid as shown in figure 1 and the results are tabulated in table 1. As already mentioned, AINA is fully complaint with the

Klonoff’s surveillance error grid standards.

Deming regression model, unlike the classical linear regression method, allows method comparison taking into account measurement errors for both methods [16]. The results in table 2 clearly indicate that no statistically significant systematic difference or proportional difference exists between the two methods-AINA and the conventional glucometer-in measuring blood glucose levels using capillary fingerstick whole blood samples. In the present study, 66 (14.2%) subjects were screened positive for diabetes using the conventional glucometer. Cohen’s kappa coefficient (κ) was calculated to measure the degree of agreement of AINA and the conventional glucometer on the presence of diabetes in the study sample. It came out to be 0.9642 (i.e., more than 0.8000) which indicates almost perfect agreement between the two devices [16]. Likewise, Lin’s concordance correlation coefficient was calculated as an index of reliability to measure the degree of agreement between the two devices on random blood glucose values. This coefficient was found to be 0.9886 (i.e., between 0.9500 to 0.9900), indicating a substantial agreement between AINA and conventional glucometer on the measured random blood glucose values [17].

Our results indicate that there is substantial clinical and statistical agreement between AINA and the conventional glucometer in measuring random blood glucose using capillary fingerstick whole blood samples. Notably, unlike conventional glucometer, AINA can additionally measure hemoglobin, lipid profile, and glycated albumin, thus making it an ideal point-of-care diagnostic device for diabetes prevention and care. It is portable and can be easily used by a frontline healthcare worker with minimal training in resource-constraint settings. Markedly, diabetes is becoming increasingly prevalent among rural populations in developing countries and is significantly contributing to ever increasing health iniquity [3,4]. To this end, AINA has the potential to be a gamechanger in the primary healthcare space in marginalized settings. First and foremost, it can screen patients for diabetes in last-mile settings by measuring the blood glucose level. The clinical and statistical accuracy and reliability of such measurements have already been corroborated by the results of this study. Second, it can screen these patients for co-morbidities such as anemia and dyslipidemia at an early stage, thus preventing complex interplay of anaemia and dyslipidemia in these diabetic patients and mitigating its potential contribution to end organ damage. Third, through estimation of glycated albumin, AINA can diagnose diabetes and also guide clinicians in maintaining optimal blood glucose levels in patients already under medication.

LIMITATIONS

The objective of this study was to assess the clinical and statistical agreement between AINA and the conventional glucometer in measuring random blood glucose. It did not include estimating the diagnostic and clinical accuracy of other blood parameters that can be measured using AINA such as hemoglobin, lipid profile, and glycated albumin. Random blood glucose is the prescribed blood test for screening patients for diabetes in primary healthcare settings in India [7], and hence was prioritized in this study. Evidence suggests that India has around 43.9 million people living with undiagnosed diabetes, and a reliable and accurate measurement of random blood glucose using AINA could contribute to effective screening of patients for diabetes in far-flung areas of the country for further clinical workup [1]. However, future studies need to be undertaken to assess the reliability and accuracy of hemoglobin, lipid profile, and glycated albumin estimates generated using AINA so as to inform policy makers on the effectiveness of leveraging AINA as a

“comprehensive” diabetes prevention and care platform for the base of the pyramid population.

CONCLUSION

Our study suggests that there is substantial clinical and statistical agreement between AINA and the conventional glucometer in measuring random blood glucose. This, coupled with the fact that AINA can be additionally employed for measuring hemoglobin, lipid profile, and glycated albumin using finger-stick capillary whole blood by the frontline health workers, makes AINA a viable option to aid diabetes care and prevention at the last-mile in resource-constraint settings.

ACKNOWLEDGEMENTS

The AINA pilot was supported by funds from

Biotechnology Industry Research Assistance Council (BIRAC), Department of Biotechnology (DBT), Government of India (DBT sanction order number BFD/AO/AO5.05Q/155/17-18). The AINA scale up was done as part of the “State Consortium to Accelerate, Leverage, and Economize (SCALE)” initiative, implemented as collaboration between Wadhwani Initiative for Sustainable Healthcare (WISH) and USAID.

DISCLAIMER

The opinions or views expressed in this article are solely those of the authors and do not necessarily express the views or opinions of the organization to which the authors are affiliated.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

REFERENCES

  1. International Diabetes Federation. IDF Diabetes Atlas, Ninth Edition 2019. Available from: https://www.diabetesatlas.org/upload/resource s/material/20200106_152211_IDFATLAS9efinal-web.pdf. Accessed 2020 February.
  2. Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: results from the International Diabetes Federation Diabetes Atlas, 9th edition.

Diabetes     Res     Clin     Pract     2019      doi:

10.1016/j.diabres.2019.107843

  1. Little M, Humphries S, Patel K, Dewey C. Decoding the Type 2 diabetes epidemic in rural India. Medical Anthropology 2017;

36(2):                      96—110.                       doi:

10.1080/01459740.2016.1231676

  1. Jahagirdar SS, Bant DD, Bathija GV. Study of prevalence of diabetes mellitus in the rural areas of Hubballi, Karnataka, India. Int J Community Med Public Health 2017; 4:

104—109.             doi:              10.18203/2394-

6040.ijcmph20164720

  1. Bommer C, Heesemann E, Sagalova V, Manne-Goehler J, Atun R, Bärnighausen T, Vollmer S. The global economic burden of diabetes in adults aged 20–79 years: a cost-ofillness study. Lancet Diabetes Endocrinol

2017; 5: 423—30. doi: 10.1016/S2213-

8587(17)30097-9

  1. Schofield JD, Liu Y, Balakrishna PR, Malik RA, Soran H. Diabetes Dyslipidaemia. Diabetes Ther 2016; 7: 203—219. doi: 10.1007/s13300-016-0167-x
  2. Directorate General of Health Services, Government of India. Guidelines of the National Programme for Prevention and Control of Cancer, Diabetes, CVDs, and Stroke (NPCDCS) 2013. Available from: https://mohfw.gov.in/sites/default/files/Operat ional%20Guidelines%20of%20NPCDCS%20 %28Revised%20-%202013-17%29_1.pdf. Accessed 2020 February.
  3. Watson PF, Petrie A. Method agreement analysis: a review of correct methodology. Theriogenology 2010; 73: 1167–1179. doi: 10.1016/j.theriogenology.2010.01.003
  4. Klonoff DC, Lias C, Vigersky R, et al. The surveillance error grid. J Diabetes Sci Technol. 2014; 8(4): 658—672. doi: 10.1177/1932296814539589
  5. Simmons DA. How should blood glucose meter system analytical performance be assessed? Journal of Diabetes Science and Technology 2016; 10(1): 178—184. doi: 10.1177/1932296815598599
  6. Klonoff DC, Parkes JL, Kovatchev BP, et al. Investigation of the Accuracy of 18 Marketed Blood Glucose Monitors. Diabetes Care 2018; 41: 1681–1688. doi: 10.2337/dc17-1960
  7. Clarke WL, Cox D, Gonder-Frederick LA, Carter W, Pohl SL. Evaluating clinical accuracy of systems for self-monitoring of blood glucose. Diabetes Care 1987; 10(5): 
  1. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care

2000;          23(8):          1143-1148.           doi:

10.2337/diacare.23.8.1143

  1. Pfutzner A, Klonoff DC, Pardo S, Parkes JL. Technical aspects of the Parkes error grid. J Diabetes Sci Technol. 2013; 7(5): 1275-1281. doi: 10.1177/193229681300700517
  2. American Diabetes Association. Standards of medical care in diabetes—2015. Diabetes Care 2015; 38(suppl): S1-S93. doi: 10.2337/dc15-S001
  3. Linnet K, Boyd JC. Selection and analytical evaluation of methods – with statistical techniques. In Burtis CA, Ashwood ER, Bruns DE (eds). Tietz Textbook of Clinical Chemistry and Molecular Diagnostics (5th edn) 2012. Elsevier Saunders, St Louis, MO, pp. 201-228
  4. McBride GB. A proposal for strength-ofagreement criteria for Lin’s Concordance Correlation Coefficient. NIWA client report: HAM2005-062. NIWA Project: MOH05201. May 2005. National Institute of Water and Atmospheric Research Ltd, Hamilton, New Zealand.