Why You Should Care: Ratings impress all of us, but sometimes it makes sense to ask whether they actually measure, in any meaningful way, what’s important.
The news came our way last week that entreVIEW—this blog, at which my intermittent scribblings appear—has been named to the 2011 “Top 25 Minnesota Blawgs” list by the Minnesota State Bar Association. While doubtful as to my own contribution to this achievement, I was, of course, among the first to applaud the wisdom of the anonymous list-maker who recognized the talent of my colleagues and their far-flung, arcane, and—yes—eclectic interests, not to mention their downright likeability.
I’m a big believer in synchronicity, so it didn’t surprise me that this pronouncement was made the very day I read Malcolm Gladwell's essay regarding the business of compiling rankings, in particular as it applies to educational institutions.
How I found my way to this essay was itself synchronistic. I’ve written before (approvingly) of the writings of New York Times columnist David Brooks. Brooks recently wrote about the winners of the Sidney Awards, which—named for philosopher Sidney Hook—honor the best magazine essays each year. Lo and behold, Gladwell’s essay was among this year’s winners. I find it hard to pass up anything Gladwell writes, and the thought of reading a highly ranked piece about the questionable validity of rankings was too good to resist.
Gladwell doesn’t bash all rankings—just those that try to be all things to all people. While I’m sure he would completely endorse the methodology and results of the Bar Association’s “Top 25 Blawgs” list (well, maybe), he sure does have problems with, for example, Car and Driver’s comparisons. His point? Judging a sports car by exactly the same criteria as a minivan does not result in helpful information. “A ranking can be heterogeneous, in other words, as long as it doesn’t try to be too comprehensive. And it can be comprehensive, as long as it doesn’t try to measure things that are heterogeneous.”
If this is true of motor vehicles, it’s especially true of institutions of higher learning. Gladwell shows how fiddling with the criteria can lead to very different rankings. (For example, emphasizing cost in law school rankings results in ranking the University of Alabama in the top 10, above the fancy schools the editors of this blog attended.) And the criteria themselves—the proxies for quality used in the rankings—in reality often have a tenuous relationship to educational quality. The resulting rankings have a power well beyond what might be expected. The illusion becomes reality: “The U. S. News ratings are a self-fulfilling prophesy.”
Entrepreneurs of all stripes learn the basic lesson early: in the marketplace, perception is reality. But, as it is with taking test drives, so should it be with relying on rankings. It’s a good idea from time to time to check under the hood.
No comments :
Post a Comment