Don't put that pencil down just yet: Former SUNY provost says evidence shows we should keep the SAT.

[Thanks to Marta Mooney (Fordham University) for sending me this link.] Recently I wrote about some selective colleges who are dropping the SAT test as an application requirement: The scores are in, and it's evidence-based management: 1, standardized-test industry: 0. But Peter Salins, former Provost of the SUNY (State University of New York) system, offers up evidence that SATs are in fact good predictors of college success. At Minding the Campus, he argues passionately against dropping the SAT, saying recent decisions to do so are "empirically unfounded" and might place collegiate academic standards at risk.

In Abandoning The SAT - Fraud or Folly? Salins says "Let's begin with predictive validity. Among the countless studies done on this subject over the years, not a single one has failed to find a high correlation between SAT scores and academic performance in college, as measured by grades or persistence. ...I had my institutional research staff repeatedly review the relationship between SAT scores and academic success among our 33 baccalaureate campuses and their 200,000 + students, and found - as all the national research has confirmed - a near perfect correlation." He goes on to say "How about purported class and race biases? ...There is absolutely no evidence that such students, when admitted on other non-test criteria, have greater academic success in college than would have been predicted by their SATs."

Salins believes this trend could be the tip of an iceberg: "Frankly, if SAT defection were only limited to the rarified world of semi-elite liberal arts colleges, I would not care one way or the other. It's a free country; these are private institutions and if this is where political correctness du jour takes them, so be it. The greater potential harm in discounting SAT or comparable objective tests as admissions criteria will be visited on non-elite institutions, most of them public, that enroll the lion's share of American college students. They will drink the anti-SAT admissions Kool-Aid promoted by their richer, more 'selective' peers but won't have the resources to process and evaluate the kinds of subjective applicant characteristics that could serve as substitutes or surrogates for SATs. And, unlike their more affluent private peers, they certainly won't be able to compensate for the academic deficiencies of entering students with supplemental tutoring or counseling. I sincerely hope that SAT defection is only a fad, limited to small high-end institutions. The arguments advanced by the as yet tiny band of top colleges or universities that are abandoning the SAT are empirically unfounded, if not downright fraudulent. ...[T]he disparagement of the SAT as an admissions criterion by the top tier schools may set an unfortunate precedent for the rest, risking widespread debasement of national collegiate academic standards that are not all that high to begin with."

Comments

[chris]

Tracy-
this points to the potential misuse (and possibly abuse) of evidence. It's important to remember that evidence is really just information. It's amazing how sometimes when we wrap up information in cute statistical packages, others view the packaging as the sole indicator of how valuable the information is to them. They forget (or don't know) users still have to digest the evidence and determine the degree to which it is applicable to them.

Of course this presents a double-edged sword for trying to get managers to use "evidence." Once you tell them they have to do more than just take it at face value, managers deem it not worth the effort, or no more valuable than an anecdotal story. Unfortunately, too many managers want their job to be mindless. Sure, they say they want an intellectual challenge, but I doubt it. If you simply follow a formula, then when the project fails you can blame the formula (which is why consultants make so much money; b/c they'll provide the formula). But if you have to actually THINK, and develop a plan all on your own, then your butt is on the line should the project fail. That's a risk too many managers are unwilling to take.

I guess the point of my tangent is that these types of stories present roadblocks in any effort to advance an evidence-based approach to management. Not so much because they highlight how evidence could be misleading, but because they highlight that evidence-based management is not some plug-and-play that managers can blame if they should fail.

(Since this is an evidence-based blog, I should note that I did no research and have only anecdotal evidence to back up my claims about managers.)