U.S. agencies aren't supposed to allow political interference with scientific evidence. Good luck with that.

Evidence Soup is back in business. These past 3 months, I've been distracted by a number of things, including a move from Denver, Colorado to the San Francisco Bay Area. So, where were we?

Will "4.37 Degrees of Separation" play at the multiplex? Awhile back I wrote about recent research to test the Six Degrees of Separation theory. New evidence suggests that people are separated by an average of 4.74 degrees (only 4.37 in the U.S.). Doesn't really roll off the tongue, and I wouldn't expect Will Smith to star in a sequel. But this latest research applies only to people on Facebook; a New York Times piece reminds us "the cohort was a self-selected group, in this case people with online access who use a particular Web site".

Desperately seeking scientific integrity. Early on, President Obama launched an effort to ensure that the public can "trust the science and scientific process informing public policy decisions". His March 9, 2009 memorandum made agencies responsible for "the highest level of integrity in all aspects of the executive branch's involvement". Great idea to de-politicize science, though it's much easier said than done -- as evidenced by the nearly-two-year delay in providing "guidelines" for agencies issuing this policy. Really. Those guidelines were published December 17, 2010 [pdf here].

Oh, and meanwhile the Obama Administration was widely criticized for using sloppy science to support its moratorium on offshore drilling after the BP disaster on the Gulf Coast. It's never simple: In real life it requires weighing risks, factoring in economic impacts, and making political tradeoffs (consider this analysis of the economic impact of that ban - [pdf here]). A scientific integrity rule won't give you all the answers you need when making such complex decisions: It's surely not as simple as determining whether scientific findings were manipulated for political purposes.

As explained in a recent National Public Radio story, now we're seeing the first challenge to federal evidence-gathering under this new regime: It's directed at the Bureau of Land Management (a branch of the Department of the Interior). More about that in a moment.

Why is this so hard? It's been rough going for agencies issuing scientific integrity policies. The basics are straightforward enough: Preventing people from twisting or quashing scientific evidence. But here are some reasons why the process is so problematic.

1. Scope. There's no bright line identifying what "science" should be included in an assessment, and therefore subjected to integrity requirements. This complicates things enough when you're working with "objective" evidence. It's even trickier when you bring in fuzzier stuff, like the dismal science (economics), or risk assessment. How can we say for sure what should be considered? Our assumptions about what's important determine what evidence we recognize - whether consciously or subconsciously. Our choices are influenced by our values - but we may not be fully aware of our values, or we may not want to articulate them in a transparent way.
2. Dissemination. Each agency's integrity policy is supposed to provide for open communication, and guide how evidence is presented to the public (see the 2010 guidelines mentioned earlier). It wouldn't serve anyone to have a free-for-all; consistent, controlled dissemination can improve usefulness and understanding. But not everyone agrees on the rights and responsibilities of scientists who want to discuss their findings with the public.
3. Whistle blowing. Among the substantial hurdles is the handling of whistle blowers. (I suppose if such a policy is going to have teeth, the people who want to blow whistles need to feel they can do so without losing their heads.)
4. Transparency. Scientific groups - such as the Union of Concerned Scientists - say they still want to see external accountability under these policies. So far, investigations of misconduct are internal.

12291 all over again? Thirty years ago, President Reagan signed Executive Order 12291, requiring cost-benefit analysis for 'major' federal regulations (those expected to impact the U.S. economy by $100 million or more). Clinton issued a similar order in 1993. In theory, this should have de-politicized some agency decision-making processes. The results (or lack thereof) were the subject of my doctoral dissertation.

As with the new mandate for scientific integrity, a requirement to weigh regulatory costs against benefits leaves lots of room for interpretation, and requires value judgments. When EPA issues a rule under the Clean Air Act, it's difficult enough to estimate how many hospital visits or early deaths are caused by a particular type of airborne particulate matter. But figuring out the social costs is harder still: Requiring businesses & governments to reduce those emissions can lead to job cuts and economic loss, which themselves cause poverty and negative health impacts.

Challenging BLM's process. Citing the new scientific integrity policy, a group called Public Employees for Environmental Responsibility (PEER) has filed a complaint against the BLM, saying "The U.S. Bureau of Land Management is carrying out an ambitious plan to map ecological trends throughout the Western U.S. but has directed scientists to exclude livestock grazing as a possible factor in changing landscapes....  [O]ne of the biggest scientific studies ever undertaken by BLM was fatally skewed from its inception by political pressure.... As a result, the assessments do not consider massive grazing impacts even though trivial disturbance factors such as rock hounding are included, [and they] limit consideration of grazing-related information only when combined in an undifferentiated lump with other native and introduced ungulates (such as deer, elk, wild horses and feral donkeys)." I didn't know we had a feral ungulate problem. But I digress.

This is a good example of how choices about collecting evidence can strongly influence the results. NPR explains that the Dept. of Interior has a scientific integrity officer who is responsible for investigating allegations of political interference. I wish him Godspeed.

 

Comments

[elliptical reviews]

No offense, but i suggest admin adding a google+ button for easy share!

[Air Max sale]

support its moratorium on offshore drilling after the BP disaster on the Gulf Coast. It's never simple: In real life it requires weighing risks, factoring in economic impacts, and making political tradeoffs (consider this analysis of the econo

[elliptical trainer reviews]

Still using typepad? try wordpress, you won't go back.