People who self-diagnose and self-prescribe using the Internet could be doing themselves more harm than good, according to a study from the University of Waterloo
The researchers found that while many people feel confident they can assess the effectiveness of treatments found on the web, separating medically beneficial ones from those that are a waste of money, dubious or even harmful is not as easy as people think.
“We found that search results have a significant effect on people’s ability to make correct treatment decisions,” said Amira Ghenai, a Ph.D. candidate at Waterloo’s David R. Cheriton School of Computer Science and the lead author of the study. “In short, finding bad information is worse than having no information at all.”
The study, which was published in the proceedings of the 3rd ACM SIGIR International Conference on Theory of Information Retrieval, examines how 60 Waterloo students went about executing searches for medical treatment online. The search results participants saw were intentionally biased toward correct or incorrect information for 10 medical treatments. For each treatment, participants were shown a search page with 10 results modelled after what they would see if they had used Google or Bing. Each result had a document title, a snippet of text and a URL to a page.
Participants were asked to pretend they had a question about the effectiveness of a treatment and they were going to search online to help them answer it. Treatments that are commonly searched for were chosen, among them, is cinnamon helpful for diabetes, are insoles helpful for back pain, does caffeine help alleviate asthma.
To bias search results towards correct information, the researchers selected eight results to be correct and two to be incorrect. A correct result was a document that contained information about the efficacy of a medical treatment that was supported by medical evidence, and an incorrect result to be one that contradicted medical evidence.
To bias search results towards incorrect information the opposite was done — eight results were selected to be incorrect and two to be correct. As a control, participants were asked to answer whether a treatment was helpful, inconclusive or unhelpful without being shown any search results.
“When search results were biased towards correct information it increased the participants’ accuracy from 43 per cent to 65 per cent. But when results were biased towards incorrect information it cut the participants’ accuracy almost in half, from 43 per cent to 23 per cent,” Ghenai said.
“The influence of incorrect information is concerning because sometimes the condition for which people seek treatment is much more serious than whether insoles can alleviate back pain and the outcome of an incorrect decision can be grave.
“When search engines mix correct with incorrect information, we have shown there is potential for significant harm. And when people are desperate they might see what they want to believe.”
