Journal of Medical Statistics and Informatics

Journal of Medical Statistics and Informatics

ISSN 2053-7662
Original Research

Assessing inter-rater agreement between multiple medical instruments with heteroscedastic measurements

Isuru D. Dassanayake1,2 and Lakshika S. Nawarathna2*

*Correspondence: Lakshika S. Nawarathna

2. Department of Statistics and Computer Science, Faculty of Science, University of Peradeniya, Peradeniya 20400, Sri Lanka.

Author Affiliations

1. Department of Mathematics and Statistics, Texas Tech University, Broadway and Boston, Lubbock, TX 79409-1042, USA.


Background: In clinical medicine, agreement evaluation plays a major role in determining the compatibility and the accuracy of newly introduced methods with pre-existing methods. These methods may be assays, clinical observers, medical devices etc. It is vital to assess the compatibility and the accuracy of these newly introduced techniques because they deal with the measurements of the human body, such as blood pressure, cholesterol level, heart rate etc. In practice, agreement evaluation is carried out among two methods of measurements and deals with the data that are homoscedastic. The main objective of this study is to extend the standard mixed model to allow the error variances to depend on magnitude of measurement and evaluate agreement between multiple methods assuming the new model, taking the heteroscedasticity into account.

Methods: In order to assess the agreement, there are two typical steps in method comparison studies. The first step is to model the data using the Heteroscedastic mixed effects model. The model fitting is carried out by using two main approaches, namely the mean method and the best linear unbiased predictor method. After fitting the model for the second step, the agreement evaluation is carried out using Concordance correlation coefficient and Total deviation index.

Results: The illustrative example contained five methods of measurements and was with heteroscedastic measurements. First, the model fitting was carried out according to the two approaches and the resulting parameters were almost identical. After the model fitting, the agreement evaluation was performed. According to the values resulted from the agreement measurements, it is clear that all five methods agree sufficiently well with the reference method.

Conclusions: The proposed model can be used to model the method comparison data with heteroscedastic measurements with multiple methods of measurements as well as balanced and unbalanced data designs. Under the proposed model, the agreement evaluation methodology for comparing multiple methods is also developed taking heteroscedasticity into account.

Keywords: Agreement evaluation, best linear unbiased predictor, concordance correlation coefficient, heteroscedasticity, mixed effects model, total deviation index

ISSN 2053-7662
Volume 5
Abstract Download