Evidence Soup

Fun-with-Evidence Friday: Scientific method applied to comedy, why Jesus has a middle name, and other stuff.

Need to explain some evidence? Take a lesson from the PowerPoint comedian. I always wished scientists and technologists would use more humor to explain their message. Now there's the PowerPoint Comedian, Tim Lee, a PhD scientist who "spent years developing simulation and analytical models of population dynamics before he discovered that this bored him to tears." Here's a link to a YouTube video of him onstage (with an impressive 500,000+ views), and an example of his humor (funnier when you hear his delivery).

The Scientist sheds light on Lee's analytical approach. "In addition to incorporating science into his act, Lee also uses it offstage, applying the statistics he learned in school to analyze his development as a comedian. There are two types of errors, he explains -- type 1 and type 2. The first is when people quit when they shouldn't, and the second is when people never quit even when they really should. 'I didn't want to make the second one,' he says. So he kept a log of every set he'd ever done, noting how well the jokes did and analyzing the results. 'I figured as long as I was improving I shouldn't quit.' Seeing his progress in 'a quantitative fashion' motivated him to keep moving forward." Very cool.

Are you a power walker? I am (since my knees lost interest in running 10Ks) -- and I'm always a little embarrassed at how wildly I swing my arms when I really get going. But it seems impossible not to do it. New evidence helps explain why we swing our arms when we walk, and why it's not as inefficient as you might guess. Cosmos Magazine, The Science of Everything, does a fantastic job of summarizing the evidence from the arm-swinging study: The researchers "used simple robots and human experiments to show that arm swinging is both easy and beneficial. The movement requires little muscular effort, yet it makes walking much easier. 'This puts to rest the theory that arm swinging is a vestigial relic from our quadrupedal ancestors,' said Steven Collins, a biomechanical engineer with the University of Michigan in the USA. 'Instead, arm swinging is a sensible part of an economic gait on two legs.'"

Yahoo's Weird Science column has a nice overview of the research in Out on a limb: Arm-swinging riddle is answered. Researchers "recruited 10 volunteers, who were asked to walk with a normal swing, an opposite-to-normal swing, with their arms folded, or held by their sides.... Arm-swinging turned out to be a plus, rather than a negative, the investigators found ...requiring little torque, or rotational twist, from the shoulder muscles." Evidently, we use 12% more metabolic energy when holding our arms still when we walk, and something called "vertical ground reaction moment" goes up by 63% -- which makes us bobble and drains energy from the lower leg muscles.

Stub your toe? Swear a blue streak and you'll feel better. Scientists have new evidence that Swearing Makes Pain More Tolerable. According to Live Science, "Swearing has been around for centuries and is an almost universal human linguistic phenomenon... It taps into emotional brain centers and appears to arise in the right brain, whereas most language production occurs in the left cerebral hemisphere.... Because swearing often has an exaggerating effect that can overstate the severity of pain, the team thought that swearing would lessen a person's tolerance. As it turned out, the opposite seems to be true." Not surprisingly, this was a popular science story. As Stephen Colbert humorously explained it on the Colbert Report, "Evidently, the pain that you feel is inversely proportional to the number of middle names you give Jesus." Happy Fun-with-Evidence Friday, everyone.

Pilot project for evidence-based physical therapy reveals the difficulty of applying evidence in the real world.

A new report, Strategies to Promote Evidence-Based Practice in Pediatric Physical Therapy: A Formative Evaluation Pilot Project, shows just how hard it can be to take research from theory to practice. The authors, Joe Schreiber, Perri Stern, Gregory Marchetti, and Ingrid Provident, discovered that even well-meaning people say they have little time -- and few incentives -- to follow evidence-based practices. *Sigh.*

The article is being published in Physical Therapy Online. Here are some highlights: "The physical therapy profession has been perceived as one that bases its practice largely on anecdotal evidence.... Physical therapists have been urged to increase evidence-based practice behaviors as a means to address this perception and to enhance the translation of knowledge from research evidence into clinical practice." This project evaluated strategies for encouraging people to integrate scientific evidence into their decision-making. In this pilot project, five participants collaborated on implementing techniques to enhance their use of evidence. Outcomes were analyzed qualitatively.

Show me the evidence. I wonder how the evidence was made available to the therapists? Was it at their fingertips, and did they have access to synthesized summaries? The abstract mentions "modest self-reported improvements in evidence-based practice behaviors, such as reading journal articles and completing database searches." Since I don't know the details of the situation, I can't conclude anything -- though at first glance, this doesn't sound like the ideal way to get evidence into practitioners' hands, and to get them to actually apply it.

Findings: "The participants were able to implement several, but not all, of the strategies and made modest self-reported improvements in evidence-based practice behaviors, such as reading journal articles and completing database searches. They identified several barriers, including a lack of time, other influences on clinical decision making, and a lack of incentives for evidence-based practice activities." The researchers say participating therapists "had positive attitudes toward evidence-based practice" and that they made modest improvements. Based on their findings, they've concluded that it's "critical for the profession to continue to investigate optimal strategies to aid practicing clinicians in applying research evidence to clinical decision making."

Evidence shows that we like it when people ooze confidence. Even if they're wrong.

In May, at the Association for Psychological Science session Often in Error, Rarely in Doubt, researchers presented findings showing that "Excessive confidence in the precision of one’s knowledge is both the most robust and the least understood form of overconfidence. This symposium investigates its ultimate causes. The evidence suggests that overprecision is caused by limitations on the working capacity in human memory, conversational norms, and social pressure."

Gold star. I must say, psychologists are pretty good at explaining their evidence to us regular folk. Their research abstracts, and even some of their academic papers, are actually understandable -- so kudos to them. Presentations at the Rarely in Doubt session included:

Overconfidence and the Representation of Uncertainty (Jack B. Soll, Duke University). "We compare uncertainty about repeatable cases (e.g., a random person’s weight) with uncertainty about specific cases (e.g., Rush Limbaugh’s weight). We find that perceptions of repeatable cases are noisy but accurate. In contrast, for specific cases people underestimate the variability around one’s best guess — they are overconfident in their answers."

Competing To Be Certain (But Wrong): Social Pressure and Overprecision in Judgment (Joseph R. Radzevick and Don A. Moore, Carnegie Mellon University and University of California, Berkeley). "We find that the competitive social pressure of a market contributes to overprecision among those competing for influence. When advisors must compete with each other, we find that the confidence they claim escalates over time. This overprecision helps advisors’ sell their advice."

Here's a draft of their paper (36-page pdf). The abstract says "Overly precise judgments claim more certainty than is objectively warranted. In this paper, we investigate whether the competitive social pressure of a market contributes to overprecision among those competing for influence. We find evidence that markets do indeed exacerbate overprecision. This evidence comes from two experiments in which advisors attempt to sell their advice. In the first experiment, advisors must compete with other advice sellers. In the second, advisors and decision makers are paired. Overprecision exists in both studies, and it helps advisors’ sell their advice."

New Scientist wrote a nice recap of Radzevick and Moore's findings in Humans Prefer Cockiness to Expertise, saying "The research... shows that we prefer advice from a confident source, even to the point that we are willing to forgive a poor track record.... [In] competitive situations, this can drive those offering advice to increasingly exaggerate how sure they are. And it spells bad news for scientists who try to be honest about gaps in their knowledge."

Armed with more evidence, now the regulators are blaming traders for wildly fluctuating oil prices.

Getting a grip on the evidence about what caused our recent oil price craziness has been especially difficult. There's been rampant speculation about whether commodity trading fueled the 2008 spikes. As explained in today's Wall Street Journal, Traders Blamed for Oil Spike: CFTC Will Pin '08 Price Surge on Speculators in a Reversal From Bush Findings, the U.S. Commodity Futures Trading Commission (CFTC) intends to publish a report during August suggesting speculators played a significant role in driving wild swings in oil prices.

 "In a contentious report last year, the main U.S. futures-market regulator pinned oil-price swings primarily on supply and demand. But that analysis was based on 'deeply flawed data,' Bart Chilton, one of four CFTC commissioners, said in an interview Monday. The CFTC's new review... adds fuel to a growing debate over financial investors who bet on the direction of commodities prices by buying contracts tied to indexes. These speculators have invested hundreds of billions of dollars in contracts that were once dominated by producers and consumers who sought to hedge against oil-market volatility." [Chart from Reuters via Wall Street Journal]

Not so fast, or we'll throw you into the dark pool. Commodity exchanges are crying foul. Craig Donohue, the CEO of CME Group, a large derivatives marketplace, says "We have not seen any empirical evidence that index funds and speculators distort prices, as has been widely alleged, nor is there any proof that putting position limits on these market participants will have any positive effect on the marketplace.... We are deeply concerned that inappropriate regulation of these markets will cause market participants to move to dark pools and other unregulated markets, causing irrevocable harm to the entire U.S. economy. We look forward to having an open dialogue on this topic with the CFTC, elected officials and industry participants to ensure the continued safety and soundness of an already highly regulated financial market, which remained one of the best operating components of our economy during the credit crisis of recent months." Donohue was scheduled to appear before the CFTC today to discuss position limits for energy futures trading.

Better evidence? According to the Wall Street, last year, then-acting CFTC chairman Walter Lukken "told the House Agriculture committee that CFTC's economists 'did not find direct evidence that speculation was driving up prices.' ...In preparing its 2008 report, the CFTC sought information from swaps dealers about their off-exchange derivatives transactions." But Chilton said "data the agency gathered was incomplete, with some players providing partial or no information.... [He] said the new report will contain a more-thorough analysis of the investors in contracts tied to oil and other commodities, and reveal cases in which single traders hold massive market positions."

You're so cute when you're mad. The evidence says you're more optimistic, and more of a risk-taker, when angry.

On Science Friday this week, I happened to overhear this great research title: Portrait of The Angry Decision Maker. The SciFri show was about the science of decision making, and the podcast is worth a listen. Jennifer Lerner (Harvard) is an entertaining radio guest. She co-authored the Portrait paper along with Larissa Tiedens (Stanford). The researchers say their findings indicate that anger has different effects than other negative emotions: We take bigger risks when angry, and are more optimistic about probable outcomes.

Is anger always a bad thing? This had me wondering if anger is really a negative emotion -- I've seen business-related articles about people starting companies when motivated by anger (anger over bureaucracy at their current job, for example). There's certainly nothing wrong with keeping a clear head -- and I'm definitely not advocating TWA (Texting While Angry) -- but I believe a little anger can be a good thing. Lo and behold, when I looked at the article online, the authors were wondering the same thing, asking "whether anger should be considered a positive emotion."

The full title of the research is Portrait of The Angry Decision Maker: How Appraisal Tendencies Shape Anger's Influence on Cognition (pdf here, and html version here). It offers empirical evidence about anger's "unique relationship top judgment and decision-making". Portrait also "proposes the hypothesis that anger will be experienced as relatively unpleasant and unrewarding when reflecting back on the source of one’s anger but experienced as relatively pleasant and rewarding when looking forward." Sounds about right.

Cory Doctorow on Boing Boing liked Portrait also, saying it's a "fascinating look at the role of irrationality on decision-making, and a damning rebuttal of the idea of rationality in the marketplace".

(BTW, I love the name of Doctorow's site, Craphound.)

Getting the evidence out there: Using Web 2.0 for healthcare innovation.

I took a look at a handful of recent presentations and articles about how Web 2.0 is advancing healthcare innovation. There are some promising signals that new technologies can move evidence-based medicine forward.

Case study of Web 2.0 use by the Cochrane Library. There's a nice slide presentation embedded on the openmedicine.ca blog. They've had good results, particularly using Twitter during the swine flu outbreak to combat "bad information with Cochrane evidence".

The group Disruptive Women in Healthcare is using a blog, Twitter, Facebook, and other Web 2.0 tools. I love how they describe what they do: "We’re not managing change; We’re not thriving on chaos. We’re driving change; We’re creating chaos.… We’re disrupting the health care status quo."

Driving the Adoption of Health IT Through Innovations in Social Media. This July 16 conference included a session on Drug & Food Safety In The Age Of Social Media and Transparency. The discussion included efforts at the FDA (Food and Drug Administration) to incorporate social media tools — including "how they are working, how they are coordinated, the lessons learned, and the challenges that have arisen". They looked at how social media tools can be incorporated into efforts to warn the public about counterfeit medications and food illnesses. The conference site includes videos of the presentations.

DLC's eHealth blog has a writeup about 5 Reasons Why Twitter is Relevant to Healthcare.

Kim Hart at the Washington Post wrote about how U.S. agencies are using new technology. "When the Food and Drug Administration issued a peanut recall, it set up a Twitter account (follow at FDCrecalls [note: I think she meant @FDArecalls]). Its following grew when the agency issued recalls for pistachios and Brussels sprouts, and it now has more than 10,000 followers. "

More evidence about social networks.

Responding to my recent post about influencers in social networks (The anti-Malcolm Gladwell speaks), Rick Niederman at the Forsyth Institute sent some useful research references, saying "you may want to consider the work of Steve Borgatti (U of KY, school of business) and Tom Valente (USC medical school).... Steve has a recent Science article on this [and] Tom has 2 textbooks." Thanks, Rick.

Interesting stuff. I'm fascinated by graphs and social network analysis, especially how they pertain to dissemination of evidence and knowledge (graph theory was my favorite math course). So I looked up the Science article, Network Analysis in the Social Sciences. It's by Stephen P. Borgatti, Ajay Mehra, Daniel J. Brass, and Giuseppe Labianca (13-Feb-2009, pp 892-895). Their abstract says "For social scientists, the theory of networks has been a gold mine, yielding explanations for social phenomena in a wide variety of disciplines from psychology to economics. Here, we review the kinds of things that social scientists have tried to explain using social network analysis and provide a nutshell description of the basic assumptions, goals, and explanatory mechanisms prevalent in the field." I recommend this well-written article for anyone who wants a basic understanding of how network analysis is applied to social science.

Social capital and consequences. The authors note that a "fundamental axiom in social network research is that a node's position in a network determines in part the opportunities and constraints that it encounters, and in this way plays an important role in a node's outcomes. This is the network thinking behind the popular concept of social capital, which in one formulation posits that the rate of return on an actor's investment in their human capital (i.e., their knowledge, skills, and abilities) is determined by their social capital (i.e., their network location)."

[Fig. 4. Two illustrative ego networks. The one on the left contains many structural holes; the one on the right contains few. From Science, Network Analysis in the Social Sciences.]

Theoretical mechanisms and egos. The article reviews the current thinking on what makes things -- money, an idea, or something else -- flow from one node to another. "Perhaps the most common mechanism for explaining consequences of social network variables is some form of direct transmission from node to node." Researchers have examined various mechanisms, including adaptation, binding, and exclusion. Take the binding mechanism, for instance (it reminds me of the concept of connectedness in graph theory). "The idea is that social ties can bind nodes together in such a way as to construct a new entity whose properties can be different from those of its constituent elements. Binding is one of the mechanisms behind the popular notion of the performance benefits of 'structural holes'. Given an ego-network (the set of nodes with direct ties to a focal node, called 'ego,' together with the set of ties among members of the ego network), a structural hole is the absence of a tie among a pair of nodes in the ego network. A well-established proposition in social network analysis is that egos with lots of structural holes are better performers in certain competitive settings. The lack of structural holes around a node means that the node's contacts are 'bound' together—they can communicate and coordinate so as to act as one, creating a formidable 'other' to negotiate with. This is the basic principle behind the benefits of worker's unions and political alliances. In contrast, a node with many structural holes can play unconnected nodes against each other, dividing and conquering."

Is there a divide-and-conquer strategy for evidence-based medicine? Network analysis has implications for evidence-based medicine and other efforts to make change happen by disseminating knowledge: Should we be finding the structural holes in the network and distributing the evidence there?

[Fig 1. From Science, Network Analysis in the Social Sciences.]

By the way, the article begins with a nice historical recap that includes Jacob Moreno's 1932 'sociometry' study of girls running away from school (see above).

Where's the evidence? Workshop on writing "practice implications" for scholarly publications.

Last week I mentioned a workshop on how to do a systematic review for evidence-based management. It's happening at the Academy of Management meeting next month in Chicago. I recently learned of another evidence-related workshop: Where's the Evidence? Writing Excellent "Practice Implications" in Scholarly Publications. I can't think of a more worthwhile topic.

Here's a portion of the abstract: "Writing effective Practice Implications (PI's) that are relevant to practitioners and that translate research findings into practice is difficult, yet critical to developing an evidence-based approach to management practice, including for health care administration." Key topics will include: What is the purpose of a PI and its role in translating a body of results into practice? Insights from the practitioner's standpoint on how to make research results useful. Details: Session #266 | Saturday, Aug 8 | 1:00-3:00pm | Swissôtel Chicago, Vevey Salon 3

Follow me on Twitter: @EvidenceSoup

Evidence Soup is on Twitter. You can follow me at @EvidenceSoup
[http://twitter.com/EvidenceSoup]

New collaborative law review is making evidence free & open source.

Monday, July 13 brought the world one step closer to open collaboration on evidence. The new International Free and Open Source Software Law Review is "a collaborative legal publication aiming to increase knowledge and understanding among lawyers about Free and Open Source Software [FOSS] issues." They discuss things like copyright, licensing, patents, open standards, case law, and statutory changes.

Collaboration is refreshing. The journal is published using Open Journal Systems, open source publishing software developed by the Public Knowledge Project under the GNU General Public License. Here's a snapshot of the editorial and publishing process:

Following the open source model. Here's an example of an article in the inaugural journal issue. Collaboration Among Counsel Celebrating the Formation of a Community of Lawyers for the Advancement of Understanding of Free and Open Source Licensing and Business Models asks "How do we advance the development of sound legal constructs for FOSS without waiting for the results of litigation? Through the same sorts of collaborative development models that brought us open source software. But, because there are legitimate concerns that cause lawyers to hesitate before jumping in to a discussion thread, the development of private places for conversations among lawyers is a positive development that will result in benefits for the entire community."

I admire what these folks are doing, and hope they're successful.