Introduction

Public medical service is an essential part of either an urban or a rural region. The rapid economic growth of China lifted half a billion people out of poverty [1], but significant economic disparities have also emerged [2]. Massive developmental gaps exist between urban and rural regions [3], [4]. The vast investment in urban regions attracted most talented physicians into urban top-level hospitals. As a result, the outstanding medical resources attract patients across the nation. The patient number and guidance from experienced tutors are two important factors for improvement of the physicians' clinical ability [5]. Rural doctors might meet various unintentional obstacles to these two factors. Several continuing medical education (CME) projects implemented to address the issue and many rural physicians indeed were trained, but the benefits were not obvious. Still, there are few methods that can help rural physicians to improve their diagnostic skill efficiently and effectively.

Most rural physicians who did not graduate from reputed medical universities or colleges do not develop solid knowledge of medicine. They often could not diagnose many illnesses independently. It is very hard for them to become independent in their current work environments. Though there are some internet resources that can help them acquire quite a bit of knowledge of medicine [6], [7], rural physicians prefer on-site clinical skills training [8], [9]. One efficient way for them is to learn in a top-level hospital. However, off-the-job training for some time is nearly impossible due to shortage of staff and insufficient funds.

Practical skills are what rural doctors need most. A more useful way is to drill those skills. Unlike training, the term ‘drilling’ refers to the process where some skill is repeated many times to help an individual master it. In the consideration of rural physicians' educational background and work condition summarized from on-site survey, the study proposed a simple on-the-job drilling schema based on the integration of practical medical recordings which were used to construct digital stimulated patients (DSP) and experienced radiological doctors as tutors to improve skills in diagnostic radiology of rural physicians. In our view, the schema belongs to the area of Personal Development Training and only covers the part of skill training. In addition, it seems that our method can also be a practical part of Foundation Training, Maintenance or Refresher Training, On-the-Job Training and Induction Training. With the advance of information and communications technology (ICT), such as the Internet, the personal computer, and the smart phone [10], many medical informations can be easily acquired. Researchers have reported on methods of using E-learning and distance education through the support of ICT devices [7], [11]–[14]. In our study, rural doctors could get the DSPs and frequently discuss questions with tutors via the information technique. We thought this schema could be effective to improve the skill level of rural physicians. Diagnostic radiology is an essential test, but was not satisfactorily mastered by most rural physicians. Therefore, this study, based on the on-the-job drilling schema, was designed to make an exploratory research on improving skills in diagnostic radiology of rural physicians.

Compared to current education [15], the DSP intervention based on practical medical recordings [16] and the drilling process was designed to stimulate the real atmosphere in radiology rooms in top-level hospitals. The outcomes of this project were evaluated by learning outcomes and performance improvement [17] ascertained from the written reports from participants. These reports were graded in four areas: the appropriate formatting of the report, the accuracy of the description of the radiographic images, the accuracy of the diagnosis, and the appropriate selection of subsequent treatment for the DSP. This study established five time points (TPs) for recording the performance of participants. A generalized mixed-efforts model was used to analyze the change in the physicians' performances and to assess the impact on each part.

The drilling mode proposed in this study is a form of simulation-based medical education (SBME) [18] and could be used as a part of CME for the rural physicians. SBME has been used for several decades. It is necessary to modify SBME when it is used in different subareas [19]. This study was aimed to explore the use of this modified SBME method and evaluate its effectiveness and efficiency in the improvement of rural physicians' education [20].

Methods

Drilling scenario

Physicians in rural hospitals did not serve many patients on one weekday and most of the patients preferred to see their physicians in the morning. Therefore, a significant amount of time was indeed available while the physicians were on duty. The scenario designed in this study was not only to help rural physicians improve their diagnostic skills but also to standardize their diagnostic habit. The scenario was a part of the general clinical process in top-level hospital (Figure 1). This project was divided into 5 periods. In the first period, all participants were evaluated before the drilling training began. Then, participants in training group completed the drilling process by themselves at each period. After they diagnosed a DSP, the rural physicians could see the diagnosis results completed by a radiologist from top-level hospitals. Besides the accuracy of the diagnosis, we hoped that learning a standardized writing style and careful descriptions would also be beneficial to help avoid medical mistakes. An e-investigator also recorded the amount of time that each participant spent on drilling. If the time that a physician spent on completing the drilling exercises was less than other participants did greatly, he/she was either placed in the control group or eliminated from the trial. At each end of one period, there was a test. When the test finished, the participants in trail group and the tutors discussed any questions and difficulties regarding diagnostic radiology. The drilling schema was built on a beta platform mainly based on Microsoft Server 2005 and Microsoft Silverlight.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Drilling process and the general process: action in the dotted box occurred at some time or not, which is decided by tutors.

https://doi.org/10.1371/journal.pone.0093889.g001

Results

The participants who had missed one or more tests or who spent little time on the drilling training were expelled from the project. The final sample numbers was 15 participants in the training group and 21 in the control group. One participant was added to the control group because he did not participate in any drilling but tested himself at each TP. Test scores at each TP were summarized using descriptive statistics including means and standard deviations (SDs) or means, maxima, and minima when datasets were not in a normal distribution.

The inter-rater reliabilities for the writing style, the accuracy of the image description, the accuracy of the diagnosis, and the correct treatment were 0.78, 0.79, 0.74 and 0.74 respectively. An internal consistency analysis of the test yielded a Cranach's alpha of 0.75.

Table 1 showed the scores of the style of writing for the two groups at each TP. Scores at the first TP represented the initial level of all participants. Before the intervention started, the control group and the drilling group did not differ significantly in scores (t[34] = 0.46, p = 0.65, d = 0.16). At each subsequent TP, the drilling group significantly outperformed the control group on the style of writing, even though the control group did also show improvement (t[34] = 3.33, p<0.001, d = 2.28). The difference became less obvious at the third TP (t[34] = 2.57, p = 0.02, d = 0.91) and fourth TP (t[34] = 1.50, p = 0.17, d = 0.53). At the fifth TP, the difference between the two groups was significant (t[34] = 4.21, p<0.001, d = 1.51). Using the first-score as a covariate, the ANCOVA also showed that except 4^th TP, the drilling group significantly outperformed the control group from the 2^nd to 5^th TPs (F[1,33] = 66.5, p<0.0001; F[1,33] = 8.10, p = 0.008; F[1,33] = 1.99, p = 0.17; F[1,33] = 18.10, p<0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Scores of the two groups at five TPs.

https://doi.org/10.1371/journal.pone.0093889.t001

Table 1 also showed the scores on the accuracy of the image description for the two groups at the five times points. The groups did not differ significantly in score until the 3^th TP (t[34] = 0.77, p = 0.44, d = 0.27; t[34] = 0.87, p = 0.39, d = 0.31; t[34] = 0.27, p = 0.79, d = −0.09). In the subsequent TPs, the drilling group outperformed the control group at the 4^th TP (t[34] = 2.08, p = 0.04, d = 0.73) and 5^th TP (t[34] = 3.10, p = 0.004, d = 1.11). Using the first-score as a covariate, the ANCOVA showed that the groups did not differ significantly in score until the 4^th TP (F[1,33] = 0.12, p<0.73; F[1,33] = 0.52, p = 0.48; F[1,33] = 3.99, p = 0.054). The drilling group outperformed the control group at the last TP(F[1,33] = 3.99, p = 0.005).

Scores on the accuracy of the diagnosis were displayed in Table 1 for the two groups at the five TPs. The difference between the two groups was significantly obvious except the 1^st TP. However, the control group outperformed the drilling group at the 2^nd TP. Then he drilling group significantly outperformed the control group until the 5^th TP. Similar results were found using the ANCOVA whose covariate was the first-score.

Scores on the correct treatment selection in Table 1 showed the differences between the two groups started to become significant at the 3^rd TP. At the end of the drilling project, the scores of the drilling group were much higher than the control group, which were consistent with the results yielded by the ANCOVA which used the first-score as the covariates.

Results of the sum of scores differed through the course. The difference in the beginning was the result of the difference amassed from the 4 parts.

As Figure 2(a) showed, after the introduction of the intervention, the drilling group showed a significant increase in the style of writing (d = 1.43, p<0.001). However, a slight decrease occurred afterwards (d = −0.25, p = 0.23). As the amount of practice increased, the progress of the drilling group was evident at subsequent TPs. The control group progressed steadily on the writing style throughout the study, except at the last TP. However, their overall progresses were not far from the initial state for the control group (d = 1.49, p<0.001), compared to the drilling group (d = 2.01, p<0.001). The group × time interaction effect in the mixed-effect model was significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Results and change on a) the style of writing, b) the accuracy of the image description, c) the accuracy of the diagnosis and d) the appropriate treatment selection plotted by group over time.

https://doi.org/10.1371/journal.pone.0093889.g002

Compared to the trends of the style of writing scores, the trends on the accuracy of image description of both groups shown in Figure 2(b) were similar; both groups progressed steadily at the first four TPs but regressed at the final TP. However, the overall progress of the drilling group was better (d = 2.56, p<0.01) than the control group (d = 1.22, p<0.01). The group × time interaction effect in the mixed-effect model was less significant than for the writing counterpart.

The performance scores of both groups for the accuracy of diagnosis displayed in Figure 2(c) were much different. The changes in the control group scores progressed steadily at the first four TPs and retrogressed at the final TP. The change in the drilling group scores was not significant at the second TP (d = 0.11, p = 0.59). After that, the improvement was obvious. The final scores of the drilling group were much better (d = 12.51, p<0.0001) than the control group (d = 2.21, p<0.001). The group × time interaction effect in the mixed-effect model was significant (p<0.0001).

The trends for the correct treatment selection were similar to those for the accuracy of diagnosis (Figure 2(d)). The drilling group showed great performance at the third TP (d = 1.47, p<0.001). Though the effect of size on the values at each TP in the control group was larger than the ing group, the overall progress was not larger. The final change (d = 2.08, p<0.001) in the control group was less than that in the drilling group (d = 11.76, p<0.0001). The group × time interaction effect in the mixed-effect model was significant.

Discussion

Besides China, many other developing countries also faced the similar inequality of physician distribution [21]–[23]. The inequality is not only reflected in the physicians number but also in the diagnostic skills of physicians. Therefore, the CME of rural physicians needs additional intervention. In this study, An on-the-job drilling schema was designed for the rural physicians, junior physicians, or physicians who did not have a strong educational background. Results showed that applying the design principle based on the training schema improved rural physicians' skills in diagnostic radiology, including the style of writing reports, the accuracy of event description, the accuracy of diagnosis, and the correct treatment selection. The detailed changes in each part were different. At the initial stage, the writing style in the training group showed a considerable amount of progress. It seemed that the writing style could be improved in a short time, because it required less basic knowledge. The accuracy of image description improved the least among the four areas. The accuracy of image description requires not only the strong knowledge in medicine but also more strict knowledge in radiology terminology. Image description needs more time to show a large amount of progress.

In the early period, participants in the training group hardly made the correct diagnosis. Gradually, they started to know checking back illness condition of DSPs. Therefore, the diagnoses became more accurate. However, some physicians became overconfident; and made some mistakes. The tutors recognized the issue and helped them to rectify those problems. In cases of equivocal medical radiographic images or where the features were not obvious, the physicians did not know how to use more advanced equipment to support the diagnosis at the beginning. After the physicians did know how to use advanced equipment, such as CT, they often suggested their use in a wrong situations. Under the tutors' guidance, the physicians gradually learned the right suggestion for advanced testing. Furthermore, by combining the image information with the physical status of the patient, the physicians selected more appropriate treatments. In the other side, the amount of change on the four components in the control group was similar. The internal differences within the control group were smaller than those in the training group. Because of the limitation of local resource, physicians in the control group learned little knowledge about radiologic images. Once these images appeared in the test, most participants in the control group made mistakes. Some serious diseases might be diagnosed as less serious conditions and an incorrect therapy would be recommended. This would delay appropriate treatment, even cause medical accident.

There were several limitations to the current study. These results just illuminated the time point when participants gain more progress than counterparts did. The time span for a skilled to be learned much by one individual cannot be estimated exactly. In our trial, it is very hard to tell the minimal time span for a usual case to be learned or mastered. The study sample was small because it was difficult to persuade more rural physicians to participate in this trial for nearly one year. This study only focused skills in diagnostic radiology that helped to diagnose some pulmonary diseases, but these indices were easy to be confused. The effectiveness of these lessons would be assessed in other medical departments to further support our conclusion. Though the time spent on the drilling by the participants in the training group was recorded with help of information technique, the real situation was hard to catch. Hence, the analysis between performance and time-used was incomplete. In the tests, we also found that there were some communications among participants of two groups. It seemed that these communications might increase the scores of the control group, but an all-round progress still required a systematic and consistent training process.

In the beginning of the study, we were concerned about the lack of motivation in the rural physicians, but it was not seen in the early stages. As the physicians made some advancement, their attitudes became more positive. The positive attitudes were seen among young physicians in both groups. The assistance from the tutors was a positive factor and could not be ignored. Another motivation was the long-term support from local governments.

Besides the introduction of new concepts and techniques in medical education in the rural hospitals, a reasonable policy must be constructed in order to support the appropriate intervention. The similar viewpoints were also expressed by some rural physicians and managers of rural hospital. Highly efficiency intervention must be supported by an optimization of techniques and policies.

Acknowledgments

We appreciate the help from the students in radiology department of Shanghai Pulmonary Hospital affiliated to Tongji University and the managers in rural hospitals.