Evidence Soup

Interview Wednesday with Terri Griffith (management professor at Santa Clara University).

For today's Interview Wednesday, we talk with Terri Griffith, a Professor of Management in the Leavey School of Business at Santa Clara University. I've known Terri for several years, and admire how she strives to maintain high standards for evidence-based decision-making, while also staying realistic (as she says below, we can confidently explain about 67% of the things we study). She writes a blog called Technology and Organizations; you can follow her on Twitter @terrigriffith.

The Five Questions.
#1. What got you interested in evidence?
"My interest in the hard sciences carried over to my career in the social sciences. I give a great deal of credit to the faculty at UC Berkeley and Carnegie Mellon for helping me see the power of methods and good measurement. It probably also helps that I do most of my work in science, engineering, & software companies. They expect to make decisions with evidence - though most of the time I think they are surprised that I agree."

What types of evidence do you work with most often (medical, business research, statistics, social science, etc.)?
"Social science and business research."

What is your involvement with evidence: applying it, advocating its use, researching/developing it, synthesizing/explaining/translating it, communicating it?
"All of the above. In my teaching I try and instill an understanding for the value of evidence and some basic around how to collect it. I’ll admit it’s one of the toughest issues to convey in a general management class - but it’s also an opportunity to demonstrate how different business courses give you different approaches for collecting and evaluating evidence.

"My colleagues and I do research based on social science techniques. The underlying effort and care is sometimes a surprise to our enterprise collaborators. I recall a pharma company being taken aback that we needed contact with hundreds of teams to effectively answer a particular question. Yes, it would have been a lot of work, but so was the resulting value and quality of the answer. Unfortunately the project wasn’t being driven at a high enough level for the project to go through. I have high hopes that a culture of evidence is growing. Companies like Google and Yahoo! are doing a lot to demonstrate the value of large scale studies before making a decision. I especially love Marissa Mayer’s (Google) 'Data is Apolitical.'

"...and my colleagues and I often have to develop custom assessment tools before beginning a new project. Right now we’re working on an assessment tool for Systems Savvy."

Where do you go looking for evidence, and what types of sources do you prefer? (formally published stuff such as journals, or something less formalized?)
"Top tier peer-reviewed journals tend to be my first stop. I’m typically looking for something very specific and am likely to know the people involved and their approach to the work. I love thinking about how I used to do that in 1980 and how I do it now (a combination of Google Scholar and our University library’s resources). Gartner and Forrester are also excellent resources for demonstrating a trend or business perceptions around a topic. They have access to sources that most academics can’t parallel."

#2. On a scale of 1 to 10, where 10=‘It’s crystal clear.’ and 1=’We have no idea why things are happening.’, how would you describe the overall “state of the evidence” in your primary field?
"5 -- As I tell my students, I can give them great confidence about 67% of behavior."

Which of these situations is most common in your field?
a) Much of the evidence we need doesn’t yet exist.
b) People don't know about the evidence that is available.
c) People don't understand the available evidence well enough to apply it successfully.
d) People don’t follow the evidence because “evidence-based” is not the expectation.

"Answering in terms of how practitioners would apply my research:
a) Much of the evidence we need doesn’t yet exist. Again, hoping that we are developing a 'culture of evidence' in organizations. I look forward to the day when it is common for organizations to approach the business school to answer burning questions. This will require a combination of my colleagues and I being able to move more quickly and there being greater value placed on the answers we find.

"also d) People don’t follow the evidence because 'evidence-based' is not the expectation. Social science research methods and results are not well known. The translation from a statistical finding to a dollar amount is often not direct and unexpected."

#3. Imagine a world where people can get the evidence they need, and exchange it easily and transparently. What barriers do you believe are preventing that world from becoming a reality?
"Combination of the above. If we just take the case of academic research and think about the underlying data, we see that compatibility and standards are a big issue.  The National Science Foundation is working on this one both in terms of how they make researchers deal with data transparency and in their funding of repositories and systems for sharing in the research process and data. Barriers are coming down, just not quickly."

Where do you see technology making things better?
"I'm hoping for a 'semantic web' of data. Something where variable names wouldn’t have to be identical for data to be matched. Google is trying to help: Google Fusion Tables."

#4. How do you prefer to share evidence with people, and explain it to them? Do you have a systematic way of doing it, or is there a format that you follow?
"Depends on the audience, and for other academics, it depends on the setting. Formal presentation, looks a lot like a journal article in terms of the flow. That’s the language that people are expecting. If it’s a shorter event, then it begins to look more like a public presentation. There I go with the story and then drill down where proof is needed. Love what Decker Communication calls 'human scale.' One example had to do with the cost of a bottle of water versus tap. They present the number in terms of how many years you could drink a bottle of tap water a day before you would hit the cost of a single bottle of brand-name water."

What mistakes do you see people making when they explain evidence?
"How about the mistakes I make...? I assume my business students have the same implicit understanding of modeling that I do. I’ll start drawing a box and arrow diagram and talking about the statistical significance, and then I realize I’ve lost most of them. I think I get excited about the idea and switch into a language that researchers take years to develop. No reason to expect the audience to speak that language."

#5. What do you want your legacy to be?
"I’d like to be remembered for making systems savvy a valued organizational skill. Systems savvy is the ability to weave technology tools, organizational practices, and human capabilities together for consistently powerful organizational performance. I see systems savvy as being important on the same scale as goal setting. Most managers have a reasonable knowledge of goal setting. We have a strong base of evidence around goal setting’s value, and why it works, and how to teach it. I hope to be involved in taking systems savvy to the same place. The result would hopefully be a vast improvement in organizational performance and similar reduction in human frustration."

#6. What question do you wish we'd asked?
Terri: "When should evidence-based thinking be introduced in education?"
"In kindergarten. From math classes to current events, evidence-based thinking should be a foundation."

Chime in. Would you like to be interviewed, or do you have someone to recommend? Drop me a note at tracy AT evidencesoup DOT com.

Holy Bisphenol A, Batman! Is this stuff safe or not? Some stakeholders are guilty of evidence abuse.

It's a Mad World. With apologies to Tears for Fears: I think it's kinda funny, I think it's kinda sad, how the words we use to argue are the best we'll ever have. Public discourse about the plastic additive BPA keeps getting uglier. The misstatements, lack of transparency, and overuse of explanation points!!! are disturbing.

There's a difference between limiting BPA levels in food products and outright banning it from laptops, DVDs, and cash register receipts (yes, i said cash register receipts). But you wouldn't always know that based on what people are saying. And different agencies are taking conflicting positions on BPA: Also not helpful.

No wonder we're confused. The American Chemistry Council released a statement deploring recent actions in Canada, saying "Just days after the European Food Safety Authority (EFSA) once again confirmed that BPA is safe for use in food-contact items, Environment Canada's announcement is contrary to the weight of worldwide scientific evidence, unwarranted and will unnecessarily confuse and alarm the public. This puts Environment Canada at odds with the recent conclusions of EFSA, the U.S. Food and Drug Administration, and the German Federal Institute for Risk Assessment, all of which have concluded that BPA is safe in contact with food. The decision also appears to contradict the very recent opinion of Health Canada, which stated in August that 'the current dietary exposure to BPA through food packaging is not expected to pose a health risk to the general population, including newborns and infants.'"

Scientific publications on BPA are, not surprisingly, full of geek speak. But the European Food Safety Authority published a press release that mere mortals can understand, saying:

"Following a detailed and comprehensive review of recent scientific literature and studies on the toxicity of bisphenol A at low doses, scientists on the EFSA CEF Panel conclude they could not identify any new evidence which would lead them to revise the current Tolerable Daily Intake [TDI] for BPA of 0.05 mg/kg body weight.... The Panel also state that the data currently available do not provide convincing evidence of neurobehavioural toxicity of BPA."

We need a straightforward synthesis. I'd love to see a table, and some graphics, recapping the evidence on BPA (with links and references) and summarizing the whole debate. Can't find one (please let me know if you run across something like that... can you hear me, Evidence Chart?) Also, a consumer-friendly poster or "issue card" would be helpful next time someone from the Sierra Club or Heritage Foundation or whatever knocked on my door during dinner.

It's not all bad. I've seen reasonable discussions of BPA risk in the New Scientist. On Twitter, Joanne Manaster (@sciencegoddess) and Roger Highfield, the New Scientist editor, mentioned the recent article Bisphenol A is everywhere – is it safe? which claims that "Mounting evidence against a chemical we are exposed to daily is being ignored. What more do regulators need?" The two epidemiologist authors believe EFSA's recent action is "seemingly discounting much recent science", saying "Now new evidence has been published linking the chemical to possible health effects as an expert group of 38 researchers from around the world studying BPA issued a 'consensus statement' in the journal Reproductive Toxicology based on more than 700 studies." Though I'd appreciate the article more if it summarized relevant evidence in an easy-to-read graphic, and provided links/references.

But some is atrocious. Naively, some groups are breathlessly calling for policymakers to "ban BPA in all consumer products" and asking people to sign petitions to that effect - without presenting any evidence whatsoever. They seem unaware of the concept of acceptable (non-zero) limits, don't seem to understand the difference between a plastic food container and a DVD, and seem to ignore the benefits of BPA.

Effective use of social media. Refreshingly, the Hands Off My Plastic Stuff! initiative raises awareness, provides specifics, and references important evidence. Of course, they're on FaceBook and Twitter. They provide useful information with a light touch.

"BPA is metabolized very quickly, it has little time to circulate in our bodies and cause any effects. Moreover, those who label BPA endocrine disruptor, should take a second look at soya, which contains far higher amounts of estrogens."

And from Facebook: "There's a bunch of people who say we need to ban the plastic thing called BPA in all consumer products – all of them! – because they think it’s bad. Well I’ve got news for you. I may like the DVD Dumb and Dumber, but I don’t plan on scarfing it down any time soon. Nor do I plan on spending a lot of time licking my laptop. These people are extremists who want to ban plastic which means they’ll either wind up taking away some of my favorite stuff or making it a lot more expensive. I'm talking about stuff like CDs, Blu Ray players, ski goggles, soccer shin guards, laptops, car parts, coffee makers...."

Confronting the media's love for pseudoscience. Robert Goldberg recently wrote a column, Pseudoscience activists, trial lawyers use media to muddy debates, in the San Francisco Examiner, saying:

“Recently, EWG [Environmental Working Group] reported that BPA-containing receipts were a threat to consumers. Its researchers tested store receipts in a handful of D.C.-area stores and then, upon finding traces of BPA, announced that these receipts were dangerous to all who dare go grocery shopping. This 'study' did not even examine if BPA was absorbed into the body. That didn't stop EWG, or the media, from warning people around the world to stop taking receipts.

"Outlets ranging from Los Angeles Times to ABC to NPR's 'Science Friday' have dutifully taken stories that went viral on the Web and portrayed BPA as the biological equivalent of global warming.... [But] a recent scientific review of BPA by the European Food Safety Authority concluded there was no reason to ban BPA. The decision was highly anticipated by the scientific and regulatory community, and has large implications for European health and environmental policy. Yet, the EFSA decision received no coverage from the mainstream American media.

"BPA makes plastic more heat-resistant and transparent, and less prone to shatter. It has made medical devices safer, reduced the incidence of food-borne diseases such as botulism, and cuts down the amount of energy needed to ship thousands of products. Yet these benefits are rarely discussed.”

Challenging legislation. There's a sloppy art to presenting evidence that convinces people to sign petitions. There's a petition supporting a U.S. food safety bill, arguing that "The reason that botulism has been virtually eliminated in America is because we have perfected canning techniques using bisphenol A (BPA), which keeps food safe to eat over time. The U.S. House of Representatives overwhelmingly passed bipartisan food safety legislation in July 2009. The effort was led by Rep. John Dingell (D-MI), a champion of food safety on Capitol Hill. More than a full year later, the Senate still hasn't acted on this legislation. So what's the hold up? Sen. Dianne Feinstein (D-CA) is trying to put a ban of BPA in the bill. But BPA is what keeps us safe from botulism!"

More on the media. Jeff Stier (JeffaStier), who until recently was with the American Council on Science and Health (ACSH), mentioned the article By Reporting Bad Science As Fact, Biased Media Help Create Panics by the Hoover Institution's Henry Miller (thankfully, Dr. Miller and the Hoover Institution refrain from using lots of !!!).

"In the heated debate over the use and effects of bisphenol A (BPA), a chemical used in food packaging and hard plastic containers, Americans are being exposed to a largely phony war. BPA is a ubiquitous industrial chemical that for more than 50 years has been an important raw material in the manufacture of polycarbonate plastics used in consumer products including beverage containers, infant feeding bottles, plastic dinnerware and plastic storage containers. BPA is also used in the lining of cans of food to prevent spoilage that can lead to bacterial contamination and the risk of botulism.

Numerous regulatory agencies in the U.S. and abroad have concluded repeatedly that BPA is safe as used and has not been shown to cause health problems in adults, children, or unborn babies. They regard the very few studies suggesting possible health concerns over BPA to be inconclusive and any such concerns to be overshadowed by the risk of replacing BPA with unproven, untested — or even more dangerous — alternatives. This is 'comparative risk assessment' in the real world."

Whew! I'm worn out.

Evidence Chart keeps people honest about the evidence.

I learned about a project called Evidence Chart through Peter Kraker on Twitter. Its purpose is to help "students to glean evidence from the research literature, articulate theories, and consider whether each piece of evidence supports or undermines each theory. Working scientists find the site useful for quickly creating a compact representation of the evidence for and against competing hypotheses."

Telling the whole truth. Evidence Chart isn't a fully fledged software product - but it's worth noting their simple approach to listing all relevant research/evidence, and identifying which items do, or do not, support a hypothesis that's being considered. Each piece of evidence is ranked (on a scale of -5 to 5) according to whether it opposes or confirms. The associated blog quotes Richard Feynman on scientific integrity, from his 1974 cargo-cult science commencement address:

"If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it.... In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another."

Each Evidence Chart corresponds to a particular research question: For example, there's a chart about What is the role of sleep on hippocampus-dependent memory consolidation? (you'll need a free account to see this one - but you can view a simple example without signing in).

Competing hypotheses. The chart lists available evidence down the left-hand column, and across the top row lists competing hypotheses relevant to the research question. (In this example, the competing hypotheses are 'no role', 'passive role', 'permissive role', and 'active role'.) Also, you can view 'Dominant' descriptions, or 'Contrarian' descriptions if you want to hear what a devil's advocate would say about the evidence.

Of course, besides Evidence Chart there are many tools for ranking decision-making criteria. But few of them are about ranking evidence based on its support of a particular theory.

Researchers explain how wet dogs dry off with a good shake. Evidently, the "hirsute animals rapidly oscillate their bodies to shed water droplets".

Happy Fun-with-Evidence Friday. I stumbled across a post on Dogster about scientists studying how wet dogs shake to dry off. Evidence reported in The Wet-Dog Shake describes, among other things, "conditions for drop ejection".

This fun Time Warp video examines the question: Is your dog a sports towel or a wet mop? My boy Petey is a standard poodle, which means he absorbs lots of water, and therefore qualifies as a wet mop.

(For more, see the Wired article Physics of Wet Dogs Shake Out in High-Speed Videos.) The research abstract, from Fluid Dynamics, speaks for itself (I suppose):

"The drying of wet fur is a critical to mammalian heat regulation. In this fluid dynamics video, we show a sequence of films demonstrating how hirsute animals rapidly oscillate their bodies to shed water droplets, nature's analogy to the spin cycle of a washing machine. High-speed videography and fur-particle tracking is employed to determine the angular position of the animal's shoulder skin as a function of time. X-ray cinematography is used to track the motion of the skeleton.

"We determine conditions for drop ejection by considering the balance of surface tension and centripetal forces on drops adhering to the animal. Particular attention is paid to rationalizing the relationship between animal size and oscillation frequency required to self-dry." I think that last part is geek-speak for "big dogs shake more slowly than little dogs."

Evidently, I shoulda bought 1,000 shares of JCrew when Laura Bush was still in the White House.

New evidence shows that Michelle Obama's wardrobe choices generated almost $3B in value for public companies during the year after the 2008 election. As explained on the Economix blog (New York Times), David Yermack of New York University determined that "the stock prices of 29 public companies - whose clothes Mrs. Obama wore in a total of 189 public appearances... experienced abnormally high returns. He estimates that the average value generated by any given appearance by Mrs. Obama was $14 million."

Yermack's analysis was published in the Harvard Business Review slide show The Michelle Obama Effect. "While the brands she chooses gain [2.3% in] value, investors punish a competitive sample of companies who make and sell clothes she doesn’t wear. The non-chosen ones lose 0.4% over the same time." (Evidently lots of people care about this stuff. There's a site called Mrs. O. where you can see photos of her making appearances, and find out who she's wearing.)

I'll confess I paid attention when I heard about Mrs. Obama's ensemble at the inauguration ceremony. I thought she really rocked that look (above), and deserved all the accolades. Here's a photo of Mrs. O. on the Tonight Show, emphasizing that she was wearing JCrew - not Dior - in the wake of the Palin/Neiman/Saks shopping scandal. (She wears a mix of high and low.)

Outdoing the French. Yermack also says that "Carla Can’t Compete: France's stylish first lady, Carla Bruni-Sarkozy, does not generate the same effect." Heh.

NPR ran a brief story also: First Lady's Wardrobe Sways Stock Prices. Happy Fun-with-Evidence Friday!

 

Interview Wednesday: Ben Miller (Univ of Colorado School of Medicine).

Evidence Soup is starting something new: It's called Interview Wednesday. We'll be interviewing insightful people to find out how they work with evidence, assess the "state of the evidence" in their field, and explore how things might be improved. (Would you like to be interviewed, or do you have someone to recommend? Let me know.)

First up is Ben Miller, who wants to change the game in healthcare, and who knows that relevant evidence is crucial to his efforts. (He wears many hats - and has one of the longest email signatures I've ever seen.) Ben Miller, PsyD, is Assistant Professor / Department of Family Medicine, Associate Director of Primary Care Outreach and Research, University of Colorado School of Medicine. Plus, he's Administrative Director - Collaborative Care Research Network (CCRN), and Senior Scientist - AAFP National Research Network. Follow Ben on Twitter: @miller7.

The Five Questions.
#1. What got you interested in evidence?
"Throughout my graduate training and even now, I am often the 'mental health' provider working in primary care. I have seen firsthand what happens when two systems of health care are integrated, and also witnessed the barriers in policy adopting such a comprehensive strategy to address the healthcare needs of the whole person. I watched as more and more evidence emerged onto the scene supporting the integration of mental health into primary care, but saw no policy movement to accommodate this new information.

"I began to wonder, what is it about the evidence that is not compelling enough to change or move policy? Being trained as a clinician, I had an appreciation for research and using evidence based treatments in whatever setting I was working, but it was not until I began to take a step back and look at why systems had not changed and what evidence they need to change that I realized I would likely spend the rest of my career working on collecting 'this' evidence."

What types of evidence do you work with most often (medical, business research, statistics, social science, etc.)?
"I am lucky to have the opportunity to work and learn from national leaders in practice-based research networks (PBRN). According to the Agency for Healthcare Research and Quality (AHRQ), PBRNs are based in primary care, and have been around for more than twenty years. PBRNs involve community-based clinicians and staff in specific activities designed to understand and improve primary care. 'The best PBRN efforts link relevant clinical questions with rigorous research methods in community settings to produce scientific information that is externally valid, and, in theory, assimilated more easily into everyday practice.' I love practice-based data – collected at the place where the 'magic is happening' – where it is difficult to control for all conditions and where you can really study, in a focused manner, the whole person."

What is your involvement with evidence: applying it, advocating its use, researching/developing it, synthesizing/explaining/translating it, communicating it?
"I am the co-creator and administrative director for the CCRN. I'm the principal investigator on federal two grants using the network to research models of integrating mental health and primary care.

"Building off the idea that a national network is stronger than individual practices in examining research questions (PBRN), the Collaborative Care Research Network (CCRN) was developed to implement a national, practice-based research agenda to evaluate the effectiveness of collaboration between mental health providers and primary care providers. With the emergence of the patient-centered medical home, it is clear that such integrated services are to be a prominent dimension of responding to a community, patient and families’ whole person health needs. However, there is an equally crucial need to evaluate the effectiveness of such 'collaborative care' in ways that are both useful to primary care practitioners, and answer the important questions that have and will be raised. For more info on the CCRN, go to www.aafp.org/nrn/ccrn. With the right evidence collected, we hope to influence policy so that healthcare systems can change and defragment."

Where do you go looking for evidence, and what types of sources do you prefer? 
"I prefer to see the evidence in practice. To watch transformative change on an individual level, community level or even national level, there is something very powerful that happens. Maybe it is in the measuring of it, dissecting of it, reporting of it or even the feeling of it that keeps you coming back for more. You start to look for better ways to research the idea, more novel ways to study the problem and more interesting ways to share the information with those who could benefit the most for it."

#2. On a scale of 1 to 10, where 10= ‘It’s crystal clear.’ and 1=’We have no idea why things are happening.’, how would you describe the overall “state of the evidence” in your primary field?
"Considering we are publishing the 'research agenda' for our field, I would say 4."

Which of these situations is most common in your field?
a) Much of the evidence we need doesn’t yet exist.
b) People don't know about the evidence that is available.
c) People don't understand the available evidence well enough to apply it successfully.
d) People don’t follow the evidence because “evidence-based” is not the expectation.

#3. Imagine a world where people can get the evidence they need, and exchange it easily and transparently. What barriers do you believe are preventing that world from becoming a reality?
"Fragmentation of healthcare delivery systems and competing business models of how to pay for healthcare."

Where do you see technology making things better?
"Opportunities to aggregate data through such mechanisms like all payer database. I also run a project that is working towards extracting data from electronic medical records across the country into one aggregated data set to perform comparative effectiveness research."

#4. How do you prefer to share evidence with people, and explain it to them? Do you have a systematic way of doing it, or is there a format that you follow?
"I like to tell the story of why we are investigating a particular topic. I use 'Ms. Jones' a lot to help me. Telling the story, and whether supporting it with evidence, or justifying why you did what you did is a highly effective way for me to get my message across."

What mistakes do you see people making when they explain evidence?
"Over generalizing findings, controlling for everything but the one disease condition (who really has just one problem?); and not explaining such essential pieces of information like NNT."

#5. What do you want your legacy to be?
"I’d like to be remembered for changing the way healthcare is operated and delivered.”

Chime in. Would you like to be interviewed, or do you have someone to recommend? Drop me a note at tracy AT evidencesoup DOT com.

Evidence shows genetically modified and conventional crops are good for each other: The corn is as high as an elephant's eye.

Recently published research demonstrates substantial benefits from genetically modified corn. Eryn Brown wrote about this in the LA Times story Genetically modified corn helps protect non-engineered cousins: "Planting genetically modified, pest-resistant corn can provide a halo effect — offering protection from insects to nearby corn plants that have not been engineered to kill bugs".

And the GM corn benefits from conventional crops. The modified crop is called Bt corn because it's been engineered with the bacterium Bacillus thuringiensis (Bt) to resist the European corn borer. Rather than replace entire crops with the Bt variety, scientists theorized that planting some non-Bt corn would provide "refuges" for the pest, preventing it from developing Bt resistance. This appears to have worked: Both crop types are benefiting. (At least until Mother Nature gets creative and designs another pest.)

These findings appear in the Science article Areawide Suppression of European Corn Borer with Bt Maize Reaps Savings to Non-Bt Maize Growers, by Hutchison et al. From the abstract: "Using statistical analysis of per capita growth rate estimates, we found that areawide suppression of the... corn borer is associated with Bt maize use. Cumulative benefits over 14 years are an estimated $3.2 billion for maize growers in Illinois, Minnesota, and Wisconsin, with more than $2.4 billion of this total accruing to non-Bt maize growers."

You say 3%, I say cold, crazy cash. Scientists don't always keep the big picture in mind. According to the LA Times story, "Margaret Mellon, director of the food and environment program at the Union of Concerned Scientists, which advises a cautious approach to genetically modified crops, wondered whether the savings attributed to Bt corn was enough to merit fanfare. By her estimates, the savings ran to only about 3% of the total value of the corn crop .... 'The benefits are real, but they're modest,' she said." Modest? Some of us don't think 3% of billions of dollars is modest.

And while we're talking about maize, here's an excellent poster by Jim Wark showing a wonderful corn maze here in Denver.

Actionable - and profitable - evidence? "This is the 'first time economists have estimated the benefits of Bt corn for non-Bt crops', according to study coauthor Paul Mitchell. Proving how well Bt corn works may, paradoxically, encourage farmers to plant less, not more, of the genetically modified crop in the future, in hopes that they could reap the benefits of Bt protection without having to pay for as many of the more expensive seeds."

The researchers conclude that "These results affirm theoretical predictions of pest population suppression and highlight economic incentives for growers to maintain non-Bt maize refugia for sustainable insect resistance management."

Why chairs comfort you like a good spouse, and how two newfangled devices obliterate your love handles.

For your Fun-with-Evidence Friday reading pleasure, I offer two items: Up first is a way to blast off fat without surgery (and without exercising). Followed by an analysis showing that we buy chairs using the same criteria we use when choosing life partners.

Fat blasting. Two noninvasive procedures are competing for our attention: Zerona Laser and Zeltiq CoolSculpting™. Both are FDA approved. The New York Times compared the two in Zap or Chill? Targeting Fat Without Surgery. And last month, the Wall Street Journal wrote about both devices in How Fat-Blasting Devices Work.

Zerona is a four-armed machine that zaps your belly, flanks and hips with laser beams. Dr. Alan Bauman, MD spoke favorably about the research behind Zerona in an interview with American Health and Beauty: "The surprising effects of certain types of low level laser therapy on fat cells have been the subject of extensive scientific research since the 1990's. Studies using MRI and electron microscopy confirmed the 'liquefying' effects of low level laser on fat cells. This research demonstrated the laser's ability to create temporary membrane pores allowing leakage of material out of the fat cells without harming them. Further studies led to the FDA's clearance of the use of the Erchonia LipoLASER to enhance liposuction procedures in 2004. Research completed in 2008 confirmed that Zerona using the LipoLASER is an effective stand-alone non-invasive body contouring therapy. Fat leaks from the adipocytes and is carried away by the lymphatics, eventually excreted through the liver."

The Zerona site says "Voluminous adipose tissue can appear when fatty material is present, and this, in turn, can bring about enlarged subcutaneous fat regions. Because subcutaneous fat is close to the skin's surface, laser emulsification can have a dramatic effect. The collapse of adipose arrangements have been noted after laser therapy, reducing their volume and providing effective body contouring results."

Zeltiq CoolSculpting™ works on an entirely different scientific principle. According to Dr. Lori Brightman, the device improves body contour using a technique called Cryolipolysis™.

"It is based on work originally done by Dr. Rox Anderson’s group at Wellman Labs.... The cells in our body react to heat or cold at different temperatures (sic) levels. The research performed by Zeltiq has helped us better understand what these levels are so we can achieve body contouring through fat reduction. It works by pressing the device against the area to be treated and then becoming cold enough to target only the fat cells in that area. The body reacts with an inflammatory response which causes the body to naturally dispose of the damaged fat cells."

Here's a YouTube demo of Zeltiq being used. One of the comments from the creator of this video says "The treatment costs $475 an area. Your lovehandles would be considered two areas." Heh.

Pick your chairmate carefully. As explained by Miller-McCune, "selecting a life partner is a lot like choosing a chair." This is based on analysis by the design researcher Ayca Cakmakli, who says:

"Psychologists have been investigating how we identify a good mate for years. The criteria used to make our decisions, as Cakmakli describes, are frighteningly similar to those reviewed to select products we want to bring home. The same set of core standards is pertinent in each situation since both objects and partners need to support us functionally and emotionally." According to Cakmakli, the common dimensions include attractiveness, social status, intelligence, trustworthiness, empathy, ambitious, exciting.

Happy FEF, everybody.

What we all can learn from GRADE: How to translate evidence into recommendations. And how to improve people's understanding by presenting it differently.

It's tough enough to put together relevant, scientifically valid evidence. But it's equally difficult to synthesize it and make decisions based on it. Where medicine is concerned, the GRADE Working Group does a good job of translating evidence into actionable, plain-English recommendations. (GRADE= Grading of Recommendations Assessment, Development and Evaluation.)

Something we all can learn from. GRADE isn't just a way to rate pieces of evidence, but also to explain it and recommend what to do based on that evidence. The group describes the benefit this way: "A systematic approach to grading the strength of management recommendations can minimize bias and aid interpretation of expert-created medical guidelines."

Financial outcomes are evidence, too. I'm impressed that people involved in GRADE specifically recognize outcomes other than medical ones. It's not just about getting a particular health result - it's about recommending steps that will improve the overall well-being of the patient (and maybe everyone else involved). This overlaps the concept of comparative effectiveness. And it parallels the concepts of value-based medicine and patient-centered medicine, which I've written about before.

Summary of Findings Tables. The SoF table is a key piece of the GRADE methodology: It's where the evidence is summarized. SoF tables include several key items (displayed in columns), including:

• Outcomes: Important outcomes (desirable and undesirable).
• Assumed risk: Expected burden of the outcomes.
• Corresponding risk: The burden of the outcomes after the intervention has been applied.
• Relative magnitude of effect: The risk, odds, or hazard ratio.
• Number of participants/studies addressing these outcomes. 
• Rating of the quality of evidence for each outcome.

Presentation matters! As you can see in this example, SoF tables aren't pretty. But the format is a concise, organized presentation of lots of evidence - and it improves rapid retrieval.

There's evidence that the GRADE SoF approach is an effective way to present evidence. Based on small, randomized studies of people in an evidence-based practice workshop, "inclusion of an SoF table in a review improved understanding and rapid retrieval of key findings." (Results appeared in the June 2010 Journal of Clinical Epidemiology.)

Here's a recap of those results: Participants with the SoF table were more likely to “agree” or “strongly agree” that it was easy to find results for important outcomes than people without the SoF table (68% vs. 40%). And participants with the SoF table were more likely to correctly answer two questions regarding results than those without it: (93% vs. 44%, and 87% vs. 11%). Also, participants with the SoF table spent an average of 90 seconds to find key information, compared with 4 minutes for people without the SoF.

Tara Horvath has put together a good basic intro to SoF (PowerPoint). Also, you can download the GRADEpro software tool for developing a Summary of Findings table.

Translating evidence into recommendations. The recommendation is another key piece of the GRADE approach: "Judgments about evidence and recommendations in healthcare are complex. For example, those making recommendations must decide between recommending selective serotonin reuptake inhibitors (SSRI’s) and tricyclics for the treatment of moderate depression must agree on which outcomes to consider, which evidence to include for each outcome, how to assess the quality of that evidence.... Because resources are always limited... they may also need to decide whether any incremental health benefits are worth the additional costs."

Two flavors: Strong & weak. "Recommendations to administer, or not administer, an intervention, should be based on the tradeoffs between benefits on the one hand, and risks, burden and, potentially, costs on the other." Based on the available evidence, if clinicians are very certain that benefits outweigh risks, they make a strong recommendation. Here's an example of one: "Short-term aspirin reduces the relative risk of death after myocardial infarction by approximately 25%. Aspirin has minimal side effects and very low cost. People’s values and preferences are such that virtually all patients suffering a myocardial infarction would, if they understood the choice they were making, opt to receive aspirin. Clinicians can thus offer a strong recommendation for aspirin administration in this setting."

Resources. Here are some of the many valuable resources on the GRADE methodology (these pages link to free PDFs of the articles). Both were published in BMJ during 2008:

1. Going from evidence to recommendations.
2. Incorporating considerations of resources use into grading recommendations.

 

 

Excellent. Google Knol emphasizes the importance of providing evidence to support claims.

And they said Google is making us stupid. For the first time I rated an entry on Google's Wikipedia-like Knol site. They're doing something different and admirable to gather evidence about people's evidence.

Rather than asking you to rate a Knol on a simple scale, the site asks a series of questions about the contribution you're reading. Google then uses an algorithm to compile the results into an overall rating. My favorite eval question asks if you believe the Knol cites adequate and reliable references to support its claims. Other questions include: Do you trust this person as an expert on this Knol's topic? and Do you believe this content is original and/or correctly attributed?

By the way, David Lewis has a nice Knol on cognitive load theory. And Kaveh Shojania has a nice one on evidence-based medicine - he even talks about Bayes :-)