For today's Interview Wednesday, we talk with Leart Shaka, Editor of Vaccine Times. I first wrote about him last year after discovering his blog. As a self-described layman, he's created an impressive repository of information about vaccines - and is an example of what one passionately curious person can do. (His journey also illustrates the pathetic state of reliable, user-friendly evidence: In a perfect world, his efforts wouldn't be needed because the appropriate resources would have already been available.)
Leart tweets as @skepdude and also @VaccineTimes. Since I first wrote about his project, he's put together a team and launched Vaccine Times. He explains why he has put so much effort into the project: "The need became apparent to me as I was verifying a claim made by a Generation Rescue report. The claim was straightforward: the US mandates 36 vaccines by the age of 5, while the average of another 30 countries was only 18. The claim itself does not sound like much, count the doses and compare. Nevertheless, I soon found that after a couple of hours of work, finding information from reliable sources, compiling it, verifying the calculations etc., just weren’t enough. If it was that time-consuming to verify a simple claim like that, I wondered, how much more time-consuming is it to verify the other claims?" Amen to that!
#1. What got you interested in evidence?
I don’t think there was any one thing, or any one event, that got me interested in evidence. I think all people are interested in evidence, and I am not different from the rest of humanity in that regard. The differences, however, between people have to do with the kind of evidence they require, and how consistent they are at applying these requirements.
As a skeptic I subscribe to the “extraordinary claims require extraordinary evidence” line of thinking. Believers of woo-woo on the other hand set a very low bar for their side, while at the same time demanding an almost impossibly high bar from the other side. To them the only criteria for evidence is “does it support my belief?” instead of “is this evidence reliable?” That is why an anti-vaccine proponent will accept an anecdote that supports his belief that vaccines are dangerous, while at the same time dismissing a mountain of scientific evidence that goes against that belief.
What types of evidence do you work with most often (medical, business research, statistics, social science, etc.)?
Since I decided to focus my skeptical activities to countering the anti-vaccine misinformation, I have been dealing mostly with medical evidence, in the form of scientific studies about vaccines. However, the science behind vaccines is not the only focus of The Vaccine Times; we also deal with suffering caused by vaccine-preventable diseases, and raising the public’s awareness about the dangers of not vaccinating. As such we also rely on information from government sources such as the CDC, the World Health Organization website, articles from news websites, other pro-health websites, blogs etc.
What is your involvement with evidence: Applying it, advocating its use, researching/developing it, synthesizing/explaining/translating it, communicating it?
I am not a scientist, as such my involvement with evidence falls more in the communication aspect, limited of course by my non-expert understanding of it. The main goal of The Vaccine Times is to publicize, to advertise so to speak, the science-based evidence about vaccines to the laypublic. However, we try to do that in a way that is easily accessible. That is why we need to rely on other kinds of evidence, besides the scientific one, such as the ones I mentioned in the previous question (news items, other websites and blogs).
Where do you go looking for evidence, and what types of sources do you prefer? (formally published stuff such as journals, or something less formalized?)
That depends on what I am trying to communicate to the public. If for example, I am summarizing a new study about vaccines, I go straight to the source, the actual study. In those cases I prefer not to rely on a news item describing the study, or a press release, I want to go straight to the primary source. In general, questions about the science, should be settled by looking at the scientific studies.
If on the other hand, I simply want to communicate a recent development, say an outbreak of an infectious disease, then I rely on news stories from various sources on the web. If the information about the outbreak is available in a government website such as the CDC, or a state’s Health Department website, I prefer to get the information there. If that is not available, I rely on news items from the web, which I read carefully to try and find out what their primary source is, and try to follow their primary source to ensure that the reporting on the news item was correct. For example, if the article says : “According to the Minnesota Health Department”, that is a clue for me to go to the Minnesota Health Department web page and do a search for the topic in question.
Same goes for blog entries. I follow their links to try to get to the primary source. Many times a blog entry will link to a news article, which in turn I have to peruse for its source of information and so on. Determining if a source of information is reliable, if it is “good enough”, is as much of an art as it is a science. You get better the more you do it. The basic idea is that you always want to go to the primary source of the information, if you can. If it is not possible for whatever reason, then secondary sources will have to do, but our reliance on such sources has to decrease accordingly.
#2. On a scale of 1 to 10, where 10= ‘It’s crystal clear.’ and 1=’We have no idea why things are happening.’, how would you describe the overall “state of the evidence” in your primary field?
I have chosen to spend my time on the vaccination issue. The rating I guess depends on the kind of question being asked. If the question is “Do vaccines reduce incidence of the infectious diseases they are meant to protect against?”, I would say the rating would be a 9.999999. In that case the science is as crystal clear as it can be that yes, vaccines reduce incidence considerably. However, in science there’s always a margin of error; there is always a bit of uncertainty for every result; I don’t think you can ever get a 10 in matters of science.
On the other hand, I may not be the best person to answer this question. I am not an expert in vaccines, so any answer I give is limited to my layman’s understanding of the evidence.
#3. Imagine a world where people can get the evidence they need, and exchange it easily and transparently. What barriers do you believe are preventing that world from becoming a reality? (data incompatibility, lack of research skills, information overload, lack of standard ways of presenting evidence, lack of motivation to follow evidence-based practices, ...)
Information overload is the main problem I see. Scientific knowledge has exploded over the past couple hundred years, and it is almost impossible for a layperson to properly understand the science behind any issue, be it vaccines, global warming or what have you. There are literally thousands of studies done on vaccines. Some of them are good, some bad. First of all, one needs to be able to access all of them, read them all. Then, one has to be able to distinguish the reliable, from the less reliable, from the unreliable studies. Then one has to somehow summarize the information on the first two and figure out where that totality of evidence is pointing. This is a tremendous task.
This process requires large amounts of time and expertise. Most laypeople, including myself, don’t have either, so at best we get an educated “feel” for the verdict of all this evidence, but we have to be clear: our “feel” does not trump expertize. A layperson who has spent 20 years looking at an issue is not at par with an expert who has spent 20 years looking at an issue, and that is where most anti-vaccine proponents stumble. They think their Google University Education is just as, if not more, reliable than an expert’s education and experience.
Where do you see technology making things better?
I can see how technology can make information, scientific evidence, more accessible. In fact the Internet has already done so. With a few clicks I can find the abstract of any study I want, if not the full article. On the other hand we don’t yet have a technology that can distinguish between the reliable, less reliable, and unreliable sources of evidence. That is still up to the individual; and that is where we have a problem. There is nothing technology can do to make someone a better researcher. It’s a tool, and as all other tools if used properly it can be extremely useful, but in the hands of someone who does not understand how to use it, it becomes useless, and sometimes even dangerous.
#4. How do you prefer to share evidence with people, and explain it to them? Do you have a systematic way of doing it, or is there a format that you follow?
That depends on who you’re sharing with, what kind of evidence you’re sharing, and what your goal is. In general, I think, one has to tailor the format to the audience. There is no simple answer, no one format that can apply to all. You must communicate differently to a parent, than you do to a rabid anti-vaccine proponent, than you do to a science-minded individual, because all 3 care about different things. They are interested in different things, and if your goal is to make a difference, to persuade one person one way or another, one speech is not going to apply to all 3 scenarios.
In general, the idea is to understand the point the other person is making; if they are correct you want to reinforce that point by adding your evidence and if they are wrong you have to figure out how and where they went wrong, and then figure out which piece of evidence will address that error better, and then figure out what is the best way to communicate that piece of evidence.
Again, it’s an art and one gets better with practice.
What mistakes do you see people making when they explain evidence?
The mistakes are multiple. You get things such as cherry picking, i.e. choosing only the evidence that supports your point of view while disregarding contrary evidence; drawing conclusions beyond what the evidence supports; not applying the same criteria to your side as to the other side, in a few words: all the logic 101 fallacies and errors.
#5. What do you want your legacy to be?
I’d like to be remembered for being The Vaccine Times guy; the guy who figured out how to get the pro-health message to the parents in a way that made it easy for them to digest. I would like to be able to dilute the anti-vaccine propaganda as much as possible, to help parents find their way to science-based evidence about vaccines.
Vaccines are extremely important, millions of lives have been saved with the advent of vaccines and there are some today who are pushing people away from life-saving vaccines. We owe it to the next generation to try and fight this kind of misinformation to the best of our abilities. I guess if I had to put it in a few words I’d like to be remembered as the guy who tried to do the right thing for our children.
Thanks for sharing these insights, Leart. You are truly an inspiration.
____________________________
Chime in. Would you like to be interviewed, or do you have someone to recommend? Drop me a note at tracy AT evidencesoup DOT com.