PDF
2. Department of Quality and Safety of Oral Health Care, College of Dental Sciences, Radboud University Medical Centre, Nijmegen, The Netherlands.
3. Department of Preventive and Restorative Dentistry, College of Dental Sciences, Radboud University Medical Centre, Nijmegen, The Netherlands.
4. Department of Dentistry, School of Health Sciences, University of Brasília, Brazil.
5. Department of Oral Function and Prosthetic Dentistry, College of Dental Sciences, Radboud University Medical Centre, Nijmegen, The Netherlands.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Objectives: This study aimed to test epidemiologists' performance in assessing dental caries-related conditions and treatments using the online Caries Assessment Spectrum and Treatment (CAST) instrument.
Methods: Forty epidemiologists from 24 different countries followed online training, which consisted of studying the CAST Manual and a scientific publication. After training, participants assessed a set of thirty-six photographs that showed various caries-related conditions using the CAST instrument (codes 0-7). Their performance was compared to reference scores determined by three experienced examiners who had assessed the same conditions on the extracted teeth (codes 0-6) and pictures of a clinical condition (code 7). Additionally, participants were asked to determine their degree of certainty using a Likert scale.
Results: A total of 32.5% of the epidemiologists scored more than 80% of the cases correctly. From a total of 1 440 possible scores, 72.8% were scored correctly. Among the incorrect scores, 17.8% were higher than the reference scores. Certainty increased with correctly scored cases.
Conclusion: Epidemiologists' performance for scoring caries-related conditions and treatments on clinical photographs using CAST was adequate.
Keywords: Dental caries, dental education, dental health surveys, epidemiologic methods, caries assessment spectrum and treatment (CAST)
In preparation for conducting an oral health survey, examiners need to undergo thorough training and be calibrated to ensure and safeguard the collection of reliable data. The training is usually performed according to set protocols, such as those of the World Health Organization (WHO) [1], the British Association for the Study of Community Dentistry [2] and the International Caries Detection and Assessment System (ICDAS) Coordinating Committee (ICDAS CC) [3]. These protocols usually engage an experienced epidemiologist, who supervises the training and acts as the gold standard (benchmark examiner). After the examiners have completed the training successfully, they are subjected to intra- and inter-examiner agreement tests on humans to ensure adequate levels of consistency before they start the clinical examination.
Training for the detection of dental carious lesions usually consists of a theoretical phase and a practical phase. Theoretical training targets the comprehension of the rationale for and content of the assessment method. The practical training might consist of the visual examination of extracted teeth, assessment of clinical photographs and performance of clinical oral examinations. With the increase in multi-centre studies performed within and among countries, traditional training and calibration sessions have become financially and logistically difficult to conduct. Consequently, online education has been suggested as a suitable alternative for training examiners in the preparation phase of epidemiological surveys [4].
The ICDAS CC has proposed an online e-learning programme, which is divided into seven sections and can be completed in 90 minutes [3]. The education system has been tested in two separate studies with a group of experienced dentists [4] and with a group of dental students [5]. In both studies, examiners' performance was recorded before and after they participated in the e-learning programme. There were no significant differences observed among experienced dentists [4] but an improved performance, expressed in a higher level of specificity, was shown among the group of students [5].
The Caries Assessment Spectrum and Treatment (CAST) instrument was introduced in 2011 for the assessment of dental caries-related conditions and treatment in epidemiological surveys [6]. Since then it has been validated for face, content and construct and has recorded a high level of reproducibility [7-9]. The application of the CAST instrument was compared to that of the WHO criterion and no difference was found in mean dmft/s and DMFT/S scores among 6-11 year olds [10].
In the CAST studies carried out so far, the traditional education method has been used for training and calibrating the examiners. However, for testing the face and content validity of CAST, conducted by experienced epidemiologists, an online education/consensus system (e-Delphi) was applied. Against this background, the aim of the present study was to test epidemiologists' performance in assessing dental caries-related conditions using the online CAST instrument.
The Ethical Board of the University approved this study (reference CEP-FM 001/2011). The epidemiologists were requested to fill in an informed consent form that gave them access to the e-Delphi programme. All patients whose teeth had been extracted and used in the validation study consented to these being used for research purposes.
CAST online training
A website created to host panel rounds for the face and content validation of the CAST instrument was used for this investigation [7]. A total of fifty-six international and experienced epidemiologists were invited to participate in the present study. They were requested to follow an online training programme on the usage of CAST, which consisted of studying the CAST Manual and a scientific publication on the rationale for and development of CAST [6]. The CAST Manual describes the rationale for the CAST instrument and contains the codes, related descriptions and colour photographs of extracted teeth for all the CAST codes. It also contains a Frequently Asked Questions section and other sections with instructions about how to get started using CAST, how to perform the training and calibration of examiners and how to present results. This background information was available for consultation by the epidemiologists during all phases of the study.
Study implementation
The study teeth were selected by three experienced dentists (not participants) from a group of 130 extracted teeth obtained from a public dental health clinic in Brazil. Immediately after extraction, the teeth had been stored in saline solution. Then they had been transported to the Dental School of the University, where they had been gently cleaned with hand instruments, toothbrushes and toothpaste, and stored in 0.1% thymol solution.
A total of thirty six teeth were found suitable for investigating the aim of the study. These included sixteen deciduous and twenty permanent teeth, which covered twenty-one occlusal, ten proximal and five buccal surfaces. Because it is more difficult to distinguish between CAST code 0 and CAST code 3, teeth with these scores were over-represented in the sample, which further consisted of all other CAST codes (Table 1) except CAST codes 7 and 8. These codes refer to abscess/fistulae and to missing teeth due to dental caries, respectively, which cannot be diagnosed from extracted teeth. In order to complete the assessment of the CAST codes (0-7), two clinical photographs of this condition (code 7) were added to the images of sampled teeth. Code 8 was excluded, as it requires the clinical examination to be correctly detected. Teeth were photographed by an experienced staff member of the Dental School, using a Nikon D70 digital camera (Tokyo, Japan), Sigma Lens 105 mm (Ronkonkoma, New York), under the same lighting conditions. The images were standardized for size, color and contrast, using Adobe Photoshop® CS5.
Table 1 : The codes and descriptions of the hierarchical CAST instrument [10].
Online assessment
Three weeks were allocated for the completion of the online training and the assessment of the photographs. One week was dedicated to training the epidemiologists, while two weeks were allocated for scoring the photographs.
One week after the epidemiologists had completed the CAST online training, they were asked to assess the set of thirty-six photographs using CAST (Table 1). In addition to this exercise, they were requested to express the degree of certainty of the score selected using a Likert scale that ranged from 1 (strongly uncertain) to 4 (very certain). The reference CAST scores were determined by three experienced examiners, who examined the 36 teeth visually and independently. Any difference in score was discussed among the examiners until consensus was reached.
Data analysis
Data analysis was performed by a biostatistician using the software IBM SPSS for Windows, version 20.0 (Chicago, IL, USA). The reference scores were compared to the individual CAST score decided by each of the epidemiologists. Their performance was determined by calculating the percentage of correctly decided scores.
A total of forty epidemiologists examined the photographs. They came from twenty-four countries: Australia, Belgium, Brazil, Chile, China, Czech Republic, Egypt, Finland, Germany, Indonesia, Ireland, Italy, Kenya, Kuwait, Nigeria, South Africa, Spain, Sweden, Switzerland, Tanzania, Thailand, The Netherlands, Turkey and The United States of America. Analyzing epidemiologists' performance revealed that thirteen of them (32.5%) scored more than 80% of the cases correctly, twentyone of them (52.5%) scored between 77.8% and 61.1% correctly and six epidemiologists (15%) scored less than 60% of the cases correctly. The epidemiologist who performed best scored 94.4% of the cases correctly and in 5.6% of the cases scored lower than the reference score. The epidemiologist who performed poorest scored 50.0% of the cases correctly and over-scored in 41.7% and under-scored in 8.3% of the cases, compared to the reference scores.
From the maximum possible 1440 scores, 72.8% were correctly assessed whilst in 9.4% of the cases a lower and in 17.8% of the cases a higher score than the reference scores was obtained. It was observed that epidemiologists had a tendency to score higher than the reference scores: seven epidemiologists (17.5%) scored more than 30% of the cases higher than the reference scores.
The data were also analyzed for the percentage of correctly scored CAST codes. Table 2 shows the percentage of correctly scored CAST codes. CAST code 1 had the poorest and code 6 the best performance.
Table 2 : Percentage of correct scores by CAST code.
From a total of thirty-six cases, six were less than 50% correctly scored. Two of these cases were related to teeth that had the occlusal surface sealed with glass-ionomer (code 1); one case was related to a restoration associated with an enamel carious lesion (Figure 1); one case concerned a tooth with fluorosis; one case concerned an internal caries-related discoloration in dentine (code 4, Figure 2)and the last case was related to CAST code 6.
Figure 1 : Restoration associated with an enamel carious lesion that was scored less than 50% correctly.
Figure 2 : Internal caries-related discolouration in dentine that was scored less than 50% correctly.
Table 3 shows the percentage of correctly scored CAST codes by level of certainty. As long as the level of certainty increased, the percentage of correct scores also increased. Epidemiologists scored 38.1% of the cases correctly but accompanied by a 'not certain' level, 'almost certain' 40.3%, 'certain' 65.4% and 'very certain' 82.1% of the cases correctly. Table 4 shows the percentage of correctly scored CAST codesby level of certainty determined by the epidemiologists.
Table 3 : Percentage of correctly scored CAST codes by level of certainty as determined by epidemiologists.
Table 4 : Percentage of lower, correctly and higher scored CAST codes of epidemiologists by level of certainty.
This was the first time that international epidemiologists attended CAST training online.Training was conducted during one week, which was considered adequate for them to access the content available on the website and to complete the practical exercises. The epidemiologists performed well, as 85% of them scored more than 60% of the cases correctly.
The training programme embraced theoretical content, which consisted of one scientific publication and the CAST Manual. The Manual had been constantly updated and the version included in the present study presented detailed explanations about each CAST code, the use of CAST and a section related to Frequently Asked Questions with answers that had emerged from the face and content validation exercise and from the use of CAST in epidemiological surveys [9,10].
Epidemiologists considered the online training suitable for training examiners in using CAST in epidemiological surveys [7]. They expressed no difficulties in assessing caries-related lesions and treatment from the colour photographs. This observation echoes the conclusion of a study in which colour photographs were compared to clinical oral examination in detecting carious lesions in epidemiological surveys. Colour photographs eased the assessment, which could be conducted at a convenient time and place [11].
The epidemiologists had a tendency to score higher than the reference scores and misclassification between codes was observed among epidemiologists. Misclassification was observed between two cases with sealed surfaces (code 1) that were mistaken for restorations. This distinction is difficult to see, especially as glass-ionomer was the restorative material. Another case was related to a restoration associated with an enamel carious lesion that caused confusion about scoring the restoration or the carious lesion (CAST codes 2 and 3). One case concerned a tooth with fluorosis, which would result in score 0, but was mistaken for a carious lesion. The last case was related to CAST code 6 in which pulpal involvement was not detected as clearly visible. These misclassifications have been given due attention in the CAST Manual. Measurement error is not necessarily a flaw of the assessment instrument, as scoring dental caries-related conditions with any classification system is difficult [12].
As long as the level of certainty increased, the percentage of correct scores also increased. This suggests that the participants could sense the difficulty level of the case they were scoring. In similar studies, experienced dentists tested ICDAS II (seconddigit only) before and after attending an e-learning programme [4] and reached high values of specificity and sensitivity. In contrast, dental students [5] and experienced examiners [13] reached low values of specificity. The former two studies only tested the second-digit of the ICDAS system and excluded the presence and the quality of sealants and restorations, which can be a source of errors, as shown in the present study.
For this reason, an increase in time and intensity of training is pertinent. For future studies we recommend that each examiner's performance be checked and, if necessary, the theoretical and practical training be repeated and expanded before calibration starts. For the CAST online training programme, the website will be updated regularly with relevant information and will also include interactive tools with frequently asked questions, examples of clinical epidemiological situations and a section for posing questions and comments. After that, the website should be publicized among the dental community as the authors are of the opinion that online education is a valuable tool.
The results of the present study showed that epidemiologists' performance in scoring caries-related conditions on clinical photographs using CAST scores was adequate, although an increase in time and intensity of training might be advantageous.
The authors declare that they have no competing interests.
All authors planed the methodology design and development. AL de Souza Hilgert and WJM van der Sanden conducted the website. EM Bronkhorst performed the statistical analysis. AL de Souza Hilgert wrote the first draft of the manuscript. All authors interpreted the results and reviewed the manuscript equally.
We are grateful to the epidemiologists for their participation during the implementation of this study. We thank Dr. Leandro Augusto Hilgert for photographing the teeth. We extend our appreciation to Fintan Planting, for designing and operating the website. We thank Mrs Susan van Tonder for skillfully checking the text for English grammar and syntax. We thank Radboud University Nijmegen for supporting this study financially.
Received: 29 March 2016 Revised: 28 April 2016
Accepted: 01 September 2016 Published: 14 September 2016
Copyright © 2015 Herbert Publications Limited. All rights reserved.
