You are here
Harvard Study: Diagnostic Apps Are Right Only Half the Time
There’s a warning out for those who go online or to apps to figure out why they have an upset stomach, nagging cough, or occasional chest pain. Symptom checkers –– sites that ask for information and suggest a diagnosis –– are accurate only about half of the time, according to a report from Kaiser Health News.
The finding is from a Harvard Medical School study that reviewed 23 sites, such as WebMD, the Mayo Clinic, and DocResponse. One third of the sites listed the correct diagnosis as the first option for patients. Half the sites had the right diagnosis among their top three results, and 58% listed it in their top 20 suggestions.
About a third of U.S. adults use the sites, although not necessarily in place of going to the doctor. Some of the diagnostic questions are also used by nurse triage phone services.
Researchers entered the symptoms of 45 patients from vignettes used to train medical students. The Mayo Clinic’s first online diagnosis was right only 17% of the time, but had the correct diagnosis on a list of 20 in 76% of cases. Dr. John Wilkinson, who works on Mayo’s symptom checker, said the tool directs patients to medical research and prepares them to talk to their doctors.
“We’re always trying to improve, but if most of the time the correct diagnosis is included in the list of possibilities, that’s all we’re attempting to do,” he said.
The diagnosis accuracy rate for physicians is 85% to 90%. But Jason Maude, who runs a high-performing tool called Isabel, says he does not want a Web-versus-doctor showdown.
“The whole point is not to set the patient against the doctor or to replace the doctor, but to make the patient much better informed and [able] to ask the doctor much better questions, and then together they should do a much better job,” he said.
Isabel ranked well in the study, showing the correct answer more than 40% of the time in the first diagnosis and 84% in the top 20 answers. Those high results, Maude says, may be because the site lets patients type in their own description of symptoms. They might describe a “tummy ache” or “stomach cramps” rather than the more clinical choice of “abdominal pain” used by many online symptom-checker tools. And Isabel asks only two or three questions before patients describe their problem, compared with sites that ask patients to click through 20 questions — steps that Maude said may discourage use.
Clarifying how and why patients use these tools is critical, the study’s authors say. This information could reduce unnecessary office visits or inform patients as they talk with their doctors. But for some, the tools may encourage people to seek unnecessary care.
Dr. Ateev Mehrotra, one of the study’s authors, said patients used symptom checkers more than 100 million times last year –– a fact that may stun some physicians.
“While most doctors know patients are going to the Internet to search for medical advice, in terms of these symptom checkers, I’ve been surprised that few of my colleagues even knew they existed,” he said.
Source: Kaiser Health News; July 9, 2015.