Evidence…and where we find it.

During my medical school education, I was enrolled in a required course that was creatively dubbed “Evidence-Based Medicine.” I’ll be honest with you, it was one of my least favorite courses. We would sit there for an hour or two and talk about complicated terms and statistical voodoo–which bothered me, because I was paying good money to learn about the human body, the disease processes that affect it, and the medications used to treat them. Certainly not about how to critically evaluate an article from a medical journal.

It wasn’t until I needed to use these principles that I began to truly appreciate the value of what I had learned. The amount of medical information available today is astronomical, and it continues to grow at a staggering rate. There is simply no way for a physician practicing in today’s fast-paced environment to read (or even see) every new journal article that is published. Searching for relevant information is a crucial skill, and it’s why the emphasis in medical education is shifting from pure memorization to developing an ability to search effectively and review medical literature with a critical eye.

I don’t claim to be a master of evidence-based medicine. However, I do attempt to practice it and to share it with you—even if it means challenging the traditional thoughts about a particular topic. I want to empower you to see through the flaws in the links that your friends share on Facebook. It is crucial for the general population to become informed healthcare consumers—and, by doing so, to help those of us who practice medicine to improve the quality and cost-effectiveness of the care we provide. This discernment is vital for parents who want to ensure that their children receive the best possible medical care, as it will enable you to safely wade through the massive amount of pseudo-science on the internet, and to become an active participant in your child’s health.

I’ve written before about the importance of practicing evidence-based pediatrics. But what exactly, is “evidence,” and where do we find it?

Evidence comes from scientific studies–not from a Facebook post by your friend’s sister or from somebody’s blog (even mine, although I try to provide links whenever possible). These studies are usually performed by academic medical centers, pharmaceutical companies, or government agencies, then published in peer-reviewed medical journals. Most of the original articles are not available without subscriptions–which can be quite costly, and they’re also pretty tedious to read. However, the “abstracts,” or brief summaries, are freely available by searching on sites like PubMed.

There are several different designs of studies, and a design is chosen based on what the researchers are trying to prove and the resources they have to commit. These study designs vary widely in their expense and their ability to make definitive conclusions. I’ve written before about the randomized controlled trial, which is in many ways, the prototypical medical study design. But there are many other types of studies, and there is a whole field of science centered around designing them. I can’t possibly cover all of that, but there are some important points to recognize.

  • Correlation does not imply causation. Simply put, the fact that A and B tend to occur together doesn’t mean that A causes B. Heck, maybe B causes A. Or both A and B are a result of C. Or any other combination of variables could interact in a way that A and B both happen. Without a properly designed study, it’s impossible to prove a cause-and-effect relationship.
  • What works for one group may not work for another group…or a particular person. If I read an article about post-menopausal women and try to apply it to my patient population (most of whom have never heard of menopause), it won’t go well. It’s important that the study was performed on a population similar to the people we treat based on its results. It’s also important to remember that these studies reflect average results; they can’t predict exactly what will happen in a particular instance.
  • Bigger studies are [usually] better. If you flip a coin 3 times, you might get all heads (in fact, 12.5% of the time, you will). But if you flip the same coin 20,000 times, you’re going to come out pretty close to 50/50. Similarly, if the results from several small studies are compiled, we can use them to help draw useful conclusions. Beware of very small studies that try to make sweeping conclusions.
  • Evidence trumps experience. Science is always changing. That’s one of the great things about it, and it’s a strong contrast to many forms of alternative medicine that pride themselves in remaining unchanged for thousands of years. Experience is valuable, but not validating. It’s certainly possible to have been doing something the wrong way for a very long time. When it’s clear that “the way we’ve always done it” was wrong, we need to be humble enough to accept that and wise enough to adapt.
  • Sometimes, there is no evidence. I once heard someone point out that there’s never been a scientific study that showed that survival rates are higher for people who jump out of airplanes with a parachute than for those who take the leap without one. In these cases, we sometimes have to resort to something known as “common sense.”

Where Your Doctor Really Gets Information

So evidence comes from studies, but let’s be real–doctors are busy (from what I hear). Nobody sits around reading every study ever published for any medicine they might consider using–although there are some milestone studies that demand individual attention. In reality, most physicians pick up information from other sources, after it has been condensed a bit.

One of the most frequently-used clinical resources is a website called Up-to-Date. This site has articles on a wide variety of medical conditions and discusses the diagnosis, treatment, and follow-up for each of them. It’s where I usually turn for quick information about diseases I don’t see often, or to review the ones I treat every day. It’s a bit of a mixture of evidence-based medicine and “expert opinion,” which can be hit-or-miss. But overall, it’s a great resource.

There are also review journals for each medical specialty that publish periodic updates on a variety of topics. In pediatrics, this resource is Pediatrics in Review. It’s a monthly publication that summarizes recent updates on several clinical topics, and it’s a great way for busy pediatricians to stay current.

Many professional organizations publish policy statements, clinical practices guidelines, or other similar publications about common topics. These documents are created by experts in a particular field who review all of the available evidence and make recommendations. They’re published by groups like the American Academy of Pediatrics (AAP), the Infectious Disease Society of America (IDSA), or the North American Society for Pediatric Gastroenterology, Hepatology and Nutrition (NASPGHAN–which happens to be the most awkward acronym in the world). These documents are often available without a subscription.

One of the most trusted places to turn for reliable evidence is the Cochrane Library. The Cochrane Collaboration is a group that reviews all the available evidence on a given topic, evaluates its validity, and then publishes a “systematic review” that combines the results from each individual study. Because Cochrane reviews (and others like them) effectively turn multiple studies into one very large study, they are very reliable sources–the closest thing to “proof” that we have.

Those are some of the most frequently-used clinical resources. But I’ll be honest, the most frequently-used resource overall…is Google. I’ve heard people complain that their doctors are worthless because they seem to just look everything up on Google. But this isn’t true at all. Allow me clarify a few points:

  • Google is not a source of information, but a way to find it. The fact that information is located using Google (or any other search engine) does not make the information itself less legitimate.
  • There is a lot of valuable information available on websites intended for “regular people,” including [gasp] Wikipedia. It’s certainly not 100% correct, but it can get you started.
  • Your doctor’s search terms are probably different from yours. For instance, typing “patterns of antimicrobial resistance in acute otitis media” into a search engine yields much different results than typing “ear infection medicine not helping.”
  • Presumably, your doctor isn’t Googling “baby won’t stop crying and has a fever” and then reading you advice from some random mom’s comment on a forum thread.
  • Especially in primary care fields like general pediatrics, there is no way for a doctor to know everything about everything. Looking something up isn’t a sign of weakness, it’s a sign of humility and dedication to quality care.

The truth is, while you probably don’t have a subscription to the major medical journals, there is plenty of valid medical information available online—you just have to look in the right places. And if your doctor takes the time to look something up, just be glad he’s not guessing.

As always, your comments are welcomed (even if you happen to disagree). I'll get back to you as soon as I can. Please try to keep it civil--I reserve the right to delete comments that are offensive or off-topic.

3 thoughts on “Evidence…and where we find it.

  1. Hi Dr. Hayes, thanks for another insightful article. I’m surprised you didn’t mention Google Scholar – for those without access to more restricted databases, this would be your best bet for finding peer-reviewed journal articles (often you can access the full pdf). Also I think it would be helpful if you spoke a bit about the peer-review process. Most people outside of scientific disciplines don’t really know/understand how this process ensures that published work is credible, high quality, and adheres to strict standards.

    • Alicia–thanks for your comments, and great suggestions. I’ll add more about the peer review process, and Google scholar is a great suggestion as well. There’s so much to include, I feel like it could have been a book.

  2. Dear Chad,

    Thanks for this article. I am a retired radiologist working as a freelance medical editor of papers written by people whose first language is not English. I am not a statistician by any means, and did not excel in statistics as an undergraduate. That said, I am often astonished at the lack of understanding of terms such as “pretest likelihood,” “selection bias,” and even “prospective” and “retrospective” in papers I edit for MDs and PhDs.

Comments are closed.