Evidence Soup

Shattering myths and communicating the evidence: National Drug Facts Week looks at the science behind drug abuse.

The National Institute on Drug Abuse (a U.S. government agency) sponsors programs to communicate evidence about drug abuse and addiction. This is National Drug Facts Week, a campaign to get information to young people.

The organization has a NIDA for Teens site, covering the science behind drug abuse, and a blog called Sara Bellum (cute). There's also a Drug Facts Week web site and a Facebook page. NIDA is on Twitter at @NIDANews (using hashtag #drugfacts2010 for this week).

November 9th is the annual Drug Facts Chat Day, when NIDA scientists have a Web chat with teens asking questions about drugs.

What's your IQ? Anyone can test their knowledge on drug abuse by taking the Drug IQ Challenge.

For instance, "What substance is most abused by high school seniors?" Marijuana, tobacco, alcohol, or ecstasy?

There's a contest (in conjunction with the Grammy Foundation) asking young musicians to submit original music and videos celebrating healthy living, or depicting the dark side of drug abuse. Who knows, maybe someone will write something on par with The Needle and the Damage Done, my personal favorite in that category.

Young people are the audience for the Drug Facts Week information. NIDA doesn't always list references for the evidence, such as in the Drug IQ quiz (they're trying to raise awareness, not write research papers). However, the Sara Bellum blog provides plenty of specifics about the science. As an example, the post First get drug smart, then think twice. presents information in plain-language terms and then links to scientific details.
Myth: "If I smoke cigarettes now and then, I won’t get addicted."
Think twice: "Each puff of a cigarette gives a smoker about 1 to 2 milligrams of nicotine.  Although that may not seem like much, it is enough to make someone addicted." bit.ly/cvgZvd
Following that link, the tobacco addiction page provides diagrams, scientific information, and references.

More evidence that learning style theories aren't backed by empirical evidence.

Cathy Moore has lots of good ideas about instructional design, and recently wrote a nice post about the evidence on learning styles. In Learning styles: Worth our time? she recaps two in-depth reviews, both concluding that all the sound and fury might signifiy nothing (via @OliviaMitchell and @catmoore #opw on Twitter).

The evidence is weak, to put it mildly. Cathy explains that four cognitive psychologists concluded: “The contrast between the enormous popularity of the learning-styles approach within education and the lack of credible evidence for its utility is, in our opinion, striking and disturbing. If classification of students’ learning styles has practical utility, it remains to be demonstrated.” This is from Learning Styles: Concepts and Evidence [pdf], an examination of learning style research published in Psychological Science in the Public Interest.

That review asked questions like "What evidence is necessary to validate interventions based on learning styles?" One of their findings was that available studies are weak: "Although the literature on learning styles is enormous, very few studies have even used an experimental methodology capable of testing the validity of learning styles applied to education. Moreover, of those that did use an appropriate method, several found results that flatly contradict the popular meshing hypothesis.” p. 105. (Apparently "meshing" means changing your teaching style to accomodate a learning style.)

Is it all about the Benjamins? Another reason to be skeptical is the strong commercial motivation to promote a learning-style concept and present it as a pressing need: Take a look at the list of competing 'inventories' and other assessment tools at the end of this post.

And there's more. Cathy also looked at conclusions from Learning Styles and Pedagogy in Post-16 Learning: A Systematic and Critical Review [free pdf download]. This impressive and exhaustive look at learning and teaching styles asked questions like "What empirical evidence is there that models of learning styles have an impact on students’ learning?" And as Cathy explains, this massive study found "no clear evidence supporting any of the many theories about learning styles. They fault weak methodology and the commercial nature of much of the research."

From Learning Styles and Pedagogy: "One characteristic of most of the advice offered to practitioners is that it consists of logical deductions from the various theories of learning style rather than conclusions drawn from the findings of empirical research.... The one implication for practice which is repeated throughout the literature on learning styles is that it is the responsibility of teachers, tutors and managers to adapt their teaching style to accommodate the learning style of their students or staff members." p. 134.

At any rate, it would be impractical to do anything about it. More from Learning Styles and Pedagogy: "But such an unqualified exhortation is both unhelpful and unrealistic, because it could be interpreted as meaning that the teacher/tutor/manager is obliged to respond appropriately to visual and verbal learners (and perhaps haptic learners also); to inductive and deductive, reflective and active, sequential and global, conceptual and concrete learners; and to those who like working in groups as well as those who prefer learning individually. Despite the strong convictions with which these ideas are promoted, we failed to find a substantial body of empirical evidence that such strategies have been tried and found successful. Advice of this type strikes practitioners as unworkable and so it tends to remain untested."

Wow. I hadn't realized how many learning style models and assessment instruments were out there. Just browsing this list (from Learning Styles and Pedagogy) makes me doubtful there could ever be agreement on this issue, and undermines the belief that learning styles - and methods for accomodating them - could ever be endorsed empirically.

• Gregorc’s Mind Styles Model and Style Delineator (GSD)
• Dunn and Dunn’s model and instruments of learning styles
• Kogan’s classification of learning styles
• Riding’s Cognitive Styles Analysis (CSA)
• Myers-Briggs Type Indicator (MBTI); the 16 MBTI personality types
• Apter’s Motivational Style Profile (MSP)
• Jackson’s Learning Styles Profiler (LSP)
• Kolb’s Learning Style Inventory (LSI)
• Honey and Mumford’s Learning Styles Questionnaire (LSQ)
• ‘Whole brain’ learning and design considerations

Cathy's take on this is "that instead of adapting instruction to one of the gajillions of learning style theories, we should build learners’ metacognitive skills and use formative assessment."

Once again, Keith Devlin delivers a fabulous explanation of the evidence.

I'm a fan of Keith Devlin, a mathematician at Stanford who explains math - and its relevance - in plain English. Thanks to Hal Steinman (@hmsteinman) for tweeting about an article Devlin wrote, The Real Reason Why Software Engineers Need Math [pdf], in Communications of the Association for Computing Machinery (2001).

Devlin says "Software engineers often proclaim that they never use any of the mathematics they learned in college." He explains, in his direct, understandable style, that they shouldn't be looking for evidence of direct application of their math studies. It's something fuzzier: "The experience of rigorous reasoning with purely abstract objects and structures". And if it weren't for their math education, they wouldn't learn how to do that.

"As far as the use of mathematics is concerned, let me admit up front that the software engineers are dead right: They don’t use their college mathematics. They are also dead wrong.... There’s no paradox here. It comes down to what you mean by that word 'use.'"

I remember my first graph theory course (my favorite math course, and useful, since my work sometimes has me thinking about stuff like directed edges). The professor told me students often asked why calculus was a prerequisite, since it's an entirely different field. She said it's because they need "mathematical maturity" before jumping into graph theory: Well said.

Devlin is affiliated with Stanford University's excellent Center for the Study of Language and Information (CSLI). And he provides great commentaries on NPR (Saturday mornings, the Weekend Edition Saturday show). When I lived in Silicon Valley, I attended some events at CSLI: Always interesting stuff going on. (By the way, a linguist also affiliated with CSLI, Geoff Nunberg, provides commentary on NPR's Fresh Air program. He's fantastic.)

Another writeup about math. I happened to run across another article on math that I also recommend: Republic of Mathematics (@republicofmath on Twitter) talks about Richard Skemp's discussion of relational vs. instrumental understanding: Two approaches to teaching math (two subjects taught under the same name).

From the software engineering article: "Devlin's book, InfoSense: Turning Information Into Knowledge ...provides a mathematically-grounded (but math-free) analysis of the distinctions between data, information, and knowledge, what it takes to turn one into another, and how we can better manage all three." I need to order a copy.

Snappy Words, an interactive visual thesaurus, connects associated words.

Snappy Words is an interactive, visual English dictionary and thesaurus. I looked up evidence on Snappy Words. Not surprisingly, many of the associated terms are legal in nature (state's evidence, for instance). The site won't change your life, but it's fun to see how it works.

IQ, SchmIQ. Looking at long-term 'noncognitive' skills, the evidence on early childhood education looks pretty good.

It's tough to get hard evidence about teaching effectiveness and educational outcomes, so we rely too much on basic quantitative measures like grades, test scores, and graduation rates. But some of the evidence shows that education can pay off when we consider fuzzier things related to life skills.

In An IQ Test for Policy Wonks, Jonathan Cohn describes the body of research on early childhood education. I've heard about this before, and maybe you have, too: Programs like Head Start seem to help right away, and then the effect fades away -- but later in life, children who go through these programs do better than those who don't. Cohn says "Who wants to argue against spending money to help kids, even poor ones? The trouble is that it hasn't always been clear these programs work. In particular, evidence suggests that preschools and other programs for poor children have only a temporary effect: Kids show gains in IQ but, within a few years, those gains disappear."

Show me the evidence that really matters. However, IQ scores are only one form of evidence, and there are others that  matter more. I'm grateful that Cohn links to the research he describes. He refers to a new article, Investing in Our Young People on SSRN (the Social Science Research Network). The findings appear as NBER Working Paper No. w16201, which explains that "The literature features the multiplicity of skills that explain success in a variety of life outcomes. Noncognitive skills play a fundamental role in successful lives. The dynamics of skill formation reveal the interplay of cognitive and noncognitive skills, and the presence of critical and sensitive periods in the life-cycle. We discuss the optimal timing of investment over the life-cycle."

Cohn also talks about a Wired article by Jonah Lehrer, How Preschool Changes the Brain. Describing the evidence on preschool, Lehrer says the "most impressive evidence consists of a few different studies that looked at the long-term effects of early childhood education. Let’s begin with the Perry Preschool Experiment, which consisted of 123 low income African-American children from Yspilanti, Michigan.... When the children were three years old, they were randomly assigned to either a treatment group, and given a high-quality preschool education, or to a control group, which received no preschool education at all. The subjects were then tracked over the ensuing decades, with the most recent analysis comparing the groups at the age of 40. The differences, even decades after the intervention, were stark: Adults assigned to the preschool program were 20 percent more likely to have graduated from high school and 19 percent less likely to have been arrested more than five times. They got much better grades, were more likely to remain married and were less dependent on welfare programs."

Noncognitive skills are life skills. Lehrer continues: "How does preschool work its magic? Interestingly, the Perry Preschool didn’t lead to a lasting boost in IQ scores. While kids exposed to preschool got an initial bump in general intelligence, this dissipated by second grade. Instead, preschool seemed to improve performance on a variety of “non-cognitive” abilities, such as self-control, persistence and grit."

The new article, sponsored by the National Bureau of Economic Research, reconsiders this and other evidence about investing in early childhood education. If people take the long view, the evidence shows that preschool (some of it, anyway) is worth the cost.

The evidence stacks up against so-called learning styles.

Olivia Mitchell has a nice article about learning styles, and how you should apply them when developing presentations: Don't bother. But she doesn't just tell us about it: She provides hard evidence, linking to meta-analysis and other research findings about learning styles.

If you ever need to create presentations, I highly recommend reading Olivia's blog, Speaking About Presenting, or following her on Twitter @OliviaMitchell.

Concluding her post Learning Styles: What every presenter ought to know, she says the evidence "suggests that rather than concern ourselves with the preferred learning style of an audience member we should think about the modality that best suits the content. For instance most people are likely to prefer to see an organizational chart, rather than have it described to them. Some types of content are likely to be better transferred to audience members by being presented in multiple modalities."

I've been doing some research of my own on modalities and presentation formats. More about that later.

Education doesn't necessarily lead to more acceptance of the evidence.

If only it were that easy: Educate people about science, and they'll accept its findings. But humans are more complicated than that.

More educated isn't necessarily more open-minded. The Denver Post ran a column today by Chris Mooney,* describing how Science errs on skeptics' reasoning. He does a good job of explaining that "as much as the public misunderstands science, scientists misunderstand the public. In particular, they often fail to realize that a more scientifically informed public will not necessarily more frequently side with scientists."

Mooney references research showing that "Republicans who are college graduates are considerably less likely to accept the scientific consensus on climate change than those who have less education....  In other words... better information is no cure-all. People are likely to simply strain it through an ideological sieve." He makes a great point in saying "In fact, more education probably makes a global-warming skeptic more persuasive and more adept at collecting information and generating arguments sympathetic to his or her point of view." (And, I suppose, maybe more likely to feel a financial sting from regulatory programs to reduce CO₂ emissions.)

Oops. The Yucca Mountain nuclear waste disposal site in Nevada provides another example. Mooney describes an effort to "set the record straight" by running informative ads saying scientific experts agree the site is safe. This backfired in a big way: After the campaign, "Nearly a third of viewers became more resistant to the repository, and among those who were already opposed, their resolve strengthened."

This reminds me of a project I worked on years ago, when my employer was applying for an air quality permit for a natural gas processing plant (it's a clean fuel, so shouldn't encounter much community resistance -- and it didn't in this case). Naively, for our town hall meetings, I thought it would be a good idea to compare the risks from the facility to the risks of driving a car or smoking cigarettes: To educate the public, I suppose. Thankfully I learned from an expert that this is a bad idea; people tend to view risks they undertake themselves very differently than they do involuntary risks.

Showing us the evidence? Since Mooney's discussion appears in a newspaper column, he gives general references to "polling by the Pew Research Center," but provides no specifics or links to the actual evidence.

*Mooney has a book called The Republican War on Science about the political right, claiming they put politics ahead of the scientific truth. But today's column wasn't about that -- it was directed at all scientists, and the public in general.

More evidence that supposedly says the Internet makes us stupid. I'm not buying it.

So Nicholas Carr has expanded Is Google Making Us Stupid? into a book, The Shallows: What the Internet Is Doing to Our Brains. He says "Dozens of studies by psychologists, neurobiologists, and educators point to the same conclusion: When we go online, we enter an environment that promotes cursory reading, hurried and distracted thinking, and superficial learning. Even as the Internet grants us easy access to vast amounts of information, it is turning us into shallower thinkers, literally changing the structure of our brain."

There is evidence, but does it matter? Carr deserves credit for doing lots of research - he's looked at plenty of evidence in putting this together. Some evidence does show that reading hypertext online can reduce readers' comprehension, as Carr explained recently in Wired. The AV Club review puts it this way: "[T]he cognitive resources required to distinguish among links and make navigational choices draws from the same pool as the neurons required for reading, leaving less power available for analysis and the kind of 'deep reading' Carr asserts is only possible offline."

But is Carr drawing justifiable inferences based on the conclusions of various brain researchers? Is he cherry-picking the evidence that supports his thesis? I'm not joining him on his leaps of logic. And what is he predicting, exactly? Is this just a big "so what"?

People are listening... Editorial reviews on Amazon include this one: "The core of education is this: developing the capacity to concentrate. The fruits of this capacity we call civilization. But all that is finished, perhaps. Welcome to the shallows, where the un-educating of homo sapiens begins. Nicholas Carr does a wonderful job synthesizing the recent cognitive research. In doing so, he gently refutes the ideologists of progress, and shows what is really at stake in the daily habits of our wired lives: the re-constitution of our minds. What emerges for the reader, inexorably, is the suspicion that we have well and truly screwed ourselves. (Matthew B. Crawford, author of Shop Class As Soulcraft )".

...but not everyone. On Twitter, the neuroscientist @mocost summed it up this way: "Somebody please tell me how I can filter out any mention of Nicholas Carr from my internets." @mocost also reminds us that "Carr doesn't seem to realize that everything rewires the brain".

Doing the book tour thing. Carr was interviewed on NPR Wednesday - his statements didn't sound like much to me, other than lots of hand-wringing and lamentation about how we don't read long books anymore. He must remember things differently than I do -- he seems to think there was a golden age when people really, really liked to contemplate, more so than now. How does that explain the fact that people are so obsessed with the nuances of those long, long Harry Potter books?

NPR summed it up this way: Carr "says research shows that as people get better at multitasking, they 'become less creative in their thinking.'" Really? How do you measure that? I see inspired, creative thinking all around me, much more so than in the 1980's. There's a short book excerpt on NPR's site.

Why is this an either-or? In the book, Carr says: "Although it may be tempting to ignore those who suggest the value of the literary mind has always been exaggerated, that would be a mistake. Their arguments are another important sign of the fundamental shift taking place in society's attitude toward intellectual achievement. Their words also make it a lot easier for people to justify that shift - to convince themselves that surfing the Web is a suitable, even superior, substitute for deep reading and other forms of calm and attentive thought." Reading lots of books isn't necessarily the measure of contemplative thought or intellectual achievement. I'm one of those who might suggest that "the literary mind has always been exaggerated" - if I knew precisely what a "literary mind" was. And I think it's wrong for Carr to say we frown more on intellectual achievement because of the Internet; look at the rise of the geeks online (not to mention the geeks who created all this technology in the first place).

Either Carr is way off base, or my Gladwell envy has morphed into Gladwell-Carr envy. Will people read this book? Ironically, if the theory is correct, maybe they won't.

Research shows a better explanation can persuade physicians to follow new evidence.

[This is a re-post from the Explanation Science blog.] Research tells us that explaining plays a role in adoption of new medical evidence: Studies show that face-to-face meetings, where an advocate explains the specifics of evidence-based guidelines, can persuade physicians to follow them.

Explaining is the thing. An article in the May 24 Archives of Internal Medicine describes a method called 'academic detailing' that mimics techniques employed by pharmaceutical sales reps: Meeting with small groups of doctors - especially opinion leaders - to provide details of medical studies and distribute newsletters, pocket cards, exam room posters, etc. Studies of prescribing patterns indicate that this approach can influence how physicians prescribe drugs. Positive results were also reported in a 2001 study. (Details about the research appear below.)

I have some questions:

• Has technology failed in this case? Is it not possible to explain the guidelines - and persuade physicians to follow them - using online or mobile technology? Or do we simply need different technology?
• If face-to-face explanations - in the form of academic detailing - are so important, what's the best way to go about it?
• If evidence-based guidelines were put together more effectively, would this one-on-one 'explaining' still be necessary?

Research recap. Here's an overview of some research on academic detailing. The May 2010 article Impact of the ALLHAT/JNC7 Dissemination Project on Thiazide-Type Diuretic Use concludes that "Academic detailing may increase physicians' implementation of clinical trial results, thereby making prescribing more consistent with evidence." The researchers looked at data where academic detailing had reached 18,524 physicians in 1,698 venues via 147 investigator-educators - these cases involved physician visits for treatment of hypertension. Surveys showed that prescriptions for thiazide-type diuretic "increased the most in counties where academic detailing activity was the highest."

Positive results were also reported in an earlier study appearing in the Archives of Internal Medicine: The 2001 article Academic Detailing to Improve Use of Broad-Spectrum Antibiotics at an Academic Medical Center found that "Targeted one-on-one education is a practical, effective, and safe method for reducing excessive broad-spectrum antibiotic use.... In multivariable analyses... the rate of unnecessary use of the 2 target antibiotics was reduced by 41% on the intervention services compared with controls (95% confidence interval, 44%-78%; P< .001)."

Loyola Medicine published a release about the recent study: A New Way to Educate Doctors about Treatment Guidelines: Use Marketing Techniques Employed by Pharmaceutical Companies. They said that "Researchers often get frustrated when doctors fail to adopt treatment guidelines backed by the best evidence-based research. But a study published in the May 24, 2010, Archives of Internal Medicine found that using techniques similar to those employed by pharmaceutical sales reps can help persuade doctors to follow the guidelines." Newswise and PhysOrg, among others, picked up the story, declaring that academic detailing is an effective way to educate doctors.

Critics say Ohio 'evidence-based' school funding not evidence-y enough. Twisted Sister agrees.

With apologies to Twisted Sister's Dee Snider, it's time we said "We're not gonna take it. Anymore."
Years ago, when a friend was elected to the state legislature, he was surprised by the complexity and crazy politics surrounding school finance. Indeed. The fur sure is flying in Ohio, where they've adopted a so-called "evidence-based model" for school funding. State Governor Ted Strickland is hanging his hat on this, saying "we should design our education system around what works. I have embraced an evidence-based education approach that harnesses research results and applies those findings to Ohio's specific circumstances.”

What a mess. Alas, it's hard to tell where the evidence and jargon end, and the unsupported claims and political rhetoric begin. Neither the advocates nor the opponents are doing a great job of explaining the evidence. We deserve better (for my ideas on how we can do better, go here).

Are you evidenced? Have you ever been evidenced? (Sorry, Jimi.) But before we begin, a shout out to Matt Naugle for his wonderful commentary on the controversy: "This isn’t very evidence-y". (More about that later.) This whole episode should make anyone think twice before broadcasting how evidence-based they are.

Looks good on paper. No doubt a lot of effort has gone into this EBM development. The KnowledgeWorks Foundation has worked with heavy hitters such as the Bill & Melinda Gates Foundation to identify critical success factors in student education. The plan's architects, Lawrence Picus and Allan Odden, produced a document called Review and Analysis of Ohio's Evidence-Based Model [link to pdf here]. It explains that the goal is to:

"identify school-based programs and educational strategies that research has shown to improve student learning.... Evidence-based studies establish prototypical schools, estimate the resources needed at each school, aggregate school-level estimates to the district level, add central office costs for each district, and sum the district totals to estimate the total statewide costs.... The evidence-based model was developed by the authors of this report. We have worked with a total of seven states... to develop evidence-based estimates of educational costs. Two states (WY and AR) have used our model to allocate resources to schools...."

 KnowledgeWorks has launched an initiative called School Funding Matters to support EBM in Ohio. They say the plan "uses a funding mechanism... founded on what components are proven to be critical for student success and reduces the over-reliance on local property taxpayers...." Specifics illustrate how complex school finance really is: Lawmakers must balance things like the state's share of education funding, property taxes, something called residual budgeting, phantom revenue, etc.

"Doubling" the double talk? Advocates of the program have said things that make me skeptical. First, they accuse their opponents of using "rhetorical" goals. But then they say they are going to "double" student performance - repeatedly using quote marks around "doubling" - and later describe performance improvements to date as what they "rhetorically call doubling". This is not transparent, not specific, and not acceptable: "[O]ur approach focuses on resources to 'double' the performance of Ohio's students over the next 4-6 years. This goal is still ambitious, though more achievable than the rhetorical goal of educating all students to high standards embedded in... most adequacy studies.... First, we review the evidence from research and best practice on what programs work in education, i.e., produce student learning gains. Second, we study schools and districts that have dramatically increased - what we rhetorically call doubling - the level of student performance over a four- to six-year period." Really? What student performance has doubled in only 4-6 years? And what led to that improvement? They need to tell us, in plain language.

Checking references. There's so much uproar that Governor Strickland released a statement about expert validation of the EBM, including a bibliography of associated research materials "sorted for easier readability and accessibility". But it's just a list, not a transparent accounting of the hard evidence - the typical "here's the bibliography" cop-out.

Are these guys giving "evidence-based" a black eye? Maybe. Applying the "evidence-based" label to yourself implies - fairly or unfairly - that the other guy isn't. Meanwhile, the EBM architects are being criticized for not ponying up adequate evidence to substantiate their claims.

The site Flypaper ("Ideas that stick from Fordham's Education Gadfly team") offers some excellent stuff. In Creator of “evidence-based model” of school funding can’t answer tough questions, they say: "No one’s arguing that setting high goals or being data-driven isn’t smart (both are part of Race to the Top and are part of many schools’ success). But anyone professing to know how to double student achievement through 10 simple steps deserves to be grilled. Members of Ohio’s funding council asked Picus why NAEP test scores have been stagnant in Wyoming and Arkansas if these are the states that have had EBM the longest. Picus said 'this is the best research we have' and 'look, I don’t claim to know that if you do this, in four years your problems will be solved.' This, as the PowerPoint slide sports a bullet that reads: “Bottom line… double performance in 4-7 years.” (I'm still waiting for someone to tell me what "doubling performance: means.)

10-step program. For the record, the 10 steps advocated by the EBM architects are: analyze state tests to understand performance challenges; set higher goals; adopt a new curriculum; commit to data-based decision making; invest in professional development; focus class time more efficiently; provide interventions to struggling learners; create professional learning communities; empower leaders to support instructional improvement; and take advantage of external expertise.

Now I'm even more confused. An analysis on EdExcellence.net claims that the Ohio EBM would actually scuttle any modernizing efforts, and "would prop up an outdated system... that establishes funding levels based on convention rather than need, sustains institutions whether they work or not, spends money with little regard for results and holds adults accountable for compliance not results." This is according to Paul Hill of the University of Washington, director of a Center on Reinventing Public Education, Senior Fellow at Brookings, and formerly of RAND. Hill was lead author on a $6 million (Gates-funded) nationwide school study, Facing the Future: Financing Productive Schools [pdf here] - contributors included 40+ economists, lawyers, financial specialists, and education policy makers. EdExcellence.net says much of Ohio's evidence-based approach "runs counter to what the Gates report recommended. That study shows that 'schools and systems that work best, especially for poor and disadvantaged youngsters, are not all alike: they use funds, teachers, students' time, materials, and technology very differently.'"

In Show Us the Evidence, the Flypaper folks write about school finance experts Eric Hanushek and Alfred Lindseth, who claim the evidence-based approach isn't credible. (Who should we believe?)

Not "evidence-y" enough. Heh. Finally, I stumbled upon the site Right Ohio, described as "some of the ugliest stuff on the web" (which made me immediately like it, since my company is Ugly Research). It's run by a funny guy (and a Democrat, apparently) named Matt Naugle, who I took a liking to, because he said "This isn’t very evidence-y of him." about Picus when he was unable to answer some important questions about the EBM plan. (On Twitter, Matt is @rightohio.)