Dental academics have adapted a test to check radar operators' skill in spotting enemy aircraft to examine their students' clinical competence.
The receiver operating characteristics analysis system is a points scale against which radar operators can mark their certainty of whether an aircraft was present, given a still photograph from a radar screen.
The department of oral surgery, medicine and pathology in the University of Wales College of Medicine has used the system to assess students' ability to identify patients who need their wisdom teeth out.
"The analysis is relevant to medical decision making, because there is an element of uncertainty," said department head Jonathan Shepherd. "This test could be useful for assessing clinical competence and reasoning."
The system requires a gold standard against which it can be checked, said Professor Shepherd. In the plane-spotting tests, the assessors knew whether or not a plane had been there, the dental test standards were based on a consensus development conference held by the United States National Institutes of Health.
Twenty-five clinical dental students at different stages of their course were given clinical histories and X-rays of 25 patients, and asked to mark on a six-point scale how certain they were that a wisdom tooth should be removed. Immediately after this, they were given a lecture, and a week later, they were asked to assess another 25 patients.
In an article in the journal Medical Education, the Cardiff team revealed that while junior students improved after the lecture, senior students showed significantly better clinical expertise and there was little difference in their results before and after the lecture. The team concluded that clinical experience has a far greater influence than formal teaching on students' ability to plan treatment.
"Putting people in lecture theatres to relation to clinical problem solving isn't beneficial," Professor Shepherd said. "They've got to be in clinics with real patients."