With apologies to Twisted Sister's Dee Snider, it's
time we said "We're not gonna take it. Anymore."
Years ago, when a friend was elected to the state legislature, he was surprised by the complexity and crazy politics surrounding school finance. Indeed. The fur sure is flying in Ohio, where they've adopted a so-called "evidence-based model" for school funding. State Governor Ted Strickland is hanging his hat on this, saying "we should design our education system around what works. I have embraced an evidence-based education approach that harnesses research results and applies those findings to Ohio's specific circumstances.”
What a mess. Alas, it's hard to tell where the evidence and jargon end, and the unsupported claims and political rhetoric begin. Neither the advocates nor the opponents are doing a great job of explaining the evidence. We deserve better (for my ideas on how we can do better, go here).
Are you evidenced? Have you ever been evidenced? (Sorry, Jimi.) But before we begin, a shout out to Matt Naugle for his wonderful commentary on the controversy: "This isn’t very evidence-y". (More about that later.) This whole episode should make anyone think twice before broadcasting how evidence-based they are.
Looks good on paper. No doubt a lot of effort has gone into this EBM development. The KnowledgeWorks Foundation has worked with heavy hitters such as the Bill &
Melinda Gates Foundation to identify critical success factors in student education. The plan's architects, Lawrence Picus and Allan Odden, produced a document called Review and Analysis of Ohio's Evidence-Based Model [link to pdf here]. It explains that the goal is to:
"identify school-based programs and educational strategies that research has shown to improve student learning.... Evidence-based studies establish prototypical schools, estimate the resources needed at each school, aggregate school-level estimates to the district level, add central office costs for each district, and sum the district totals to estimate the total statewide costs.... The evidence-based model was developed by the authors of this report. We have worked with a total of seven states... to develop evidence-based estimates of educational costs. Two states (WY and AR) have used our model to allocate resources to schools...."
KnowledgeWorks has launched an initiative called School
Funding Matters to support EBM in Ohio. They say the plan "uses a funding mechanism... founded on what components are proven to be critical for student success and reduces the over-reliance on local property taxpayers...." Specifics illustrate how complex school finance really is: Lawmakers must balance things like the state's share of education funding, property taxes, something called residual budgeting, phantom revenue, etc.
"Doubling" the double talk? Advocates of the program have said things that make me skeptical. First, they accuse their opponents of using "rhetorical" goals. But then they say they are going to "double" student performance - repeatedly using quote marks around "doubling" - and later describe performance improvements to date as what they "rhetorically call doubling". This is not transparent, not specific, and not acceptable: "[O]ur approach focuses on resources to 'double' the performance of Ohio's students over the next 4-6 years. This goal is still ambitious, though more achievable than the rhetorical goal of educating all students to high standards embedded in... most adequacy studies.... First, we review the evidence from research and best practice on what programs work in education, i.e., produce student learning gains. Second, we study schools and districts that have dramatically increased - what we rhetorically call doubling - the level of student performance over a four- to six-year period." Really? What student performance has doubled in only 4-6 years? And what led to that improvement? They need to tell us, in plain language.
Checking references. There's so much uproar that Governor Strickland released a statement about expert validation of the EBM, including a bibliography of associated research materials "sorted for easier readability and accessibility". But it's just a list, not a transparent accounting of the hard evidence - the typical "here's the bibliography" cop-out.
Are these guys giving "evidence-based" a black eye? Maybe. Applying the "evidence-based" label to yourself implies - fairly or unfairly - that the other guy isn't. Meanwhile, the EBM architects are being criticized for not ponying up adequate evidence to substantiate their claims.
The site Flypaper ("Ideas that stick from Fordham's Education Gadfly team") offers some excellent stuff. In Creator
of “evidence-based model” of school funding can’t answer tough questions, they say: "No one’s arguing that setting high goals or being data-driven isn’t smart (both are part of Race to the Top and are part of many schools’ success). But anyone professing to know how to double student achievement through 10 simple steps deserves to be grilled. Members of Ohio’s funding council asked Picus why NAEP test scores have been stagnant in Wyoming and Arkansas if these are the states that have had EBM the longest. Picus said 'this is the best research we have' and 'look, I don’t claim to know that if you do this, in four years your problems will be solved.' This, as the PowerPoint slide sports a bullet that reads: “Bottom line… double performance in 4-7 years.” (I'm still waiting for someone to tell me what "doubling performance: means.)
10-step program. For the record, the 10 steps advocated by the EBM architects are: analyze state tests to understand performance challenges; set higher goals; adopt a new curriculum; commit to data-based decision making; invest in professional development; focus class time more efficiently; provide interventions to struggling learners; create professional learning communities; empower leaders to support instructional improvement; and take advantage of external expertise.
Now I'm even more confused. An analysis on
EdExcellence.net claims that the Ohio EBM would actually scuttle any modernizing efforts, and "would prop up an outdated system... that establishes funding levels based on convention
rather than need, sustains institutions whether they work or not, spends money with little regard for results and holds adults accountable for compliance not results." This is according to Paul Hill of the University of Washington, director of a Center on Reinventing Public Education, Senior Fellow at Brookings, and formerly of RAND. Hill was lead author on a $6 million (Gates-funded)
nationwide school study,
Facing the Future: Financing Productive Schools [
pdf here] - contributors included 40+ economists, lawyers, financial specialists, and education policy makers.
EdExcellence.net says much of Ohio's evidence-based approach "
runs counter to what the Gates report recommended. That study shows that 'schools and systems that work best, especially for poor and disadvantaged youngsters, are not all alike: they use funds, teachers, students' time, materials, and technology very differently.'"
In Show Us the Evidence, the Flypaper folks write about school finance experts Eric Hanushek and Alfred Lindseth, who claim the evidence-based approach isn't credible. (Who should we believe?)
Not "evidence-y" enough. Heh. Finally, I stumbled upon the site Right Ohio, described as "some of the ugliest stuff on the web" (which made me immediately like it, since my company is Ugly Research). It's run by a funny guy (and a Democrat, apparently) named Matt
Naugle, who I took a liking to, because he said "This isn’t very evidence-y of him." about Picus when he was unable to answer some important questions about the EBM plan. (On Twitter, Matt is @rightohio.)