First Monday has published findings from an “empirical examination of Wikipedia’s credibility” conducted by Thomas Chesney, a Lecturer in Information Systems at the Nottingham University Business School. Chesney divided participants in the study — 69 PhD students, research fellows and research assistants — into “expert” and “non-expert” groups. This meant that roughly half were asked to evaluate an article from their field of expertise while the others were given one chosen at random (short “stub” articles excluded). The surprise finding of the study is that the experts rated their articles higher than the non-experts. Ars Technica reported this as the latest shocker in the debate over Wikipedia’s accuracy, hearkening back to the controversial Nature study comparing science articles with equivalent Britannica entries.
At a first glance, the findings are indeed counterintuitive but it’s unclear what, if anything, they reveal. It’s natural that academics would be more guarded about topics outside their area of specialty. The “non-experts” in this group were put on less solid ground, confronted at random by the overwhelming eclecticism of Wikipedia — it’s not surprising that their appraisal was more reserved. Chesney acknowledges this, and cautions readers not to take this as anything approaching definitive proof of Wikipedia’s overall quality. Still, one wonders if this is even the right debate to be having.
Accuracy will continue to be a focal point in the Wikipedia discussion, and other studies will no doubt be brought forth that add fuel to this or that side. But the bigger question, especially for scholars, concerns the pedagogical implications of the wiki model itself. Wikipedia is not an encyclopedia in the Britannica sense, it’s a project about knowledge creation — a civic arena in which experts and non-experts alike can collectively assemble information. What then should be the scholar’s approach and/or involvement? What guidelines should they draw up for students? How might they use it as a teaching tool?
A side note: One has to ask whether the experts group in Chesney’s study leaned more toward the sciences or the humanities — no small question since in Wikipedia it’s the latter that tends to be the locus of controversy. It has been generally acknowledged that science, technology (and pop culture) are Wikipedia’s strengths while the more subjective fields of history, literature, philosophy — not to mention contemporary socio-cultural topics — are a mixed bag. Chesney does never tells us how broad or narrow a cross section of academic disciplines is represented in his very small sample of experts — the one example given is “a member of the Fungal Biology and Genetics Research Group (in the Institute of Genetics at Nottingham University).”
Returning to the question of pedagogy, and binding it up with the concern over quality of Wikipedia’s coverage of humanities subjects, I turn to Roy Rosenzweig, who has done some of the most cogent thinking on what academics — historians in particular — ought to do with Wikipedia. From “Can History be Open Source? Wikipedia and the Future of the Past”:
Professional historians have things to learn not only from the open and democratic distribution model of Wikipedia but also from its open and democratic production model. Although Wikipedia as a product is problematic as a sole source of information, the process of creating Wikipedia fosters an appreciation of the very skills that historians try to teach…
Participants in the editing process also often learn a more complex lesson about history writing–namely that the “facts” of the past and the way those facts are arranged and reported are often highly contested…
Thus, those who create Wikipedia’s articles and debate their contents are involved in an astonishingly intense and widespread process of democratic self-education. Wikipedia, observes one Wikipedia activist, “teaches both contributors and the readers. By empowering contributors to inform others, it gives them incentive to learn how to do so effectively, and how to write well and neutrally.” The classicist James O’Donnell has argued that the benefit of Wikipedia may be greater for its active participants than for its readers: “A community that finds a way to talk in this way is creating education and online discourse at a higher level.”…
Should those who write history for a living join such popular history makers in writing history in Wikipedia? My own tentative answer is yes. If Wikipedia is becoming the family encyclopedia for the twenty-first century, historians probably have a professional obligation to make it as good as possible. And if every member of the Organization of American Historians devoted just one day to improving the entries in her or his areas of expertise, it would not only significantly raise the quality of Wikipedia, it would also enhance popular historical literacy. Historians could similarly play a role by participating in the populist peer review process that certifies contributions as featured articles.
ben said:
> Should those who write history for a living
> join such popular history makers
> in writing history in Wikipedia?
> My own tentative answer is yes.
and we should care what _your_ answer is
for what reason again?
and why is it you quote roy rosenzweig, from
an article in “the journal of american history”?
ponder your answers to those questions a bit…
you are correct that the _central_ question
revolves not around _accuracy_, not per se.
i say it revolves around _expertise_, and who
gets _credit_ (and ultimately, a _salary_)
for saying things that _anyone_ could say…
but of course, that’s not anything that
hasn’t already been said before, is it?
well, no, not really…
people who “write history for a living”
— and “experts” in other fields too —
_will_ start contributing to wikipedia.
but it won’t be due to some sense of
“professional obligation”, or to
“raise the quality” of wikipedia so
wikipedia is “as good as possible”,
or even to “certify contributions”
as “featured articles”. not hardly.
they’ll start getting involved so as to
retain an aura of recognized expertise,
and thus continue to collect a paycheck.
-bowerbird
bowerbird,
I think you’ve got it backwards. Academics *already have* recognized expertise. There may be a larger debate to be had over the value of academic participation in culture (I’m for it), but overall, those like rosenzweig, who have created success through teaching, publications, projects, and public thinking earned their paychecks long before they step up to the wiki.
As to the question of whether academics have a role to play in the process of *Wikipedia*, I, like rosenzweig, think the answer is yes. It’s not a professional duty, but a vocational one. If you are interested in history, and interested in improving public knowledge and literacy, then it is absolutely incumbent upon you to participate. I think it’s still unclear whether academic institutions will consider Wikipedia contributions as a factor in tenure review or advancement. Until the institutions change, I don’t foresee academics making time for Wikipedia *except* out of good will.
Your view, that academics will contribute to retain an aura of expertise is either terribly cynical or extraordinarily optimistic. Cynical in the sense that academics would contribute primarily as a way to enhance their personal or professional reputations, rather than simply to participate in the exciting melee of public discourse. Extraordinarily optimistic in the sense that you consider Wikipedia to be such a powerful mechanism that detachment from the fray would harm a person’s professional advancement.
jesse said:
> I think you’ve got it backwards.
> Academics *already have* recognized expertise.
no, i’ve got it right.
yes, they’ve already got “recognized expertise”.
but with a massive horde of talented “amateurs”
now at their heels, and soon passing them by,
they will have to work, hard, to _keep_ it…
and they won’t have to “work hard to keep it”
due to a demand from the tenure committees,
but rather from the fact that if they do not,
the _world_at_large_ will leave ’em behind,
even if they _have_ tenure…
in other words, keep proving yourself by
staying on the cutting-edge, or lose out…
(anyone who has been in academia knows
there is already a lot of tenured dead wood.)
the forces of authority believe they will
continue to be in charge. they are wrong.
ideas, having attained the speed of light, are
no longer constrained by “publication lag”…
and the massive indexing of the search engines
will make it easy to sort out the “real” credit
for the ideas that have the best staying power.
no more poaching under the peer review purview.
this is neither “cynical” nor “optimistic”;
it is basic evolutionary pressure at work.
and the place you see it happening _now_,
already, in 2006, is political commentary.
sure most of it is still happening where it
always has — i.e., in the mass media — but
the bloggers have served notice that they
are a force that simply must be considered.
play the game at their speed, or not at all.
likewise, when the “amateur historians” start
doing better history than the “professionals”
— and everyone involved admits that it’s so —
the professionals will have to join the battle
if they want to _retain_ any credibility at all.
perhaps this is easier for me to see as a poet,
where talented amateurs are now _routinely_
leaving the recognized “experts” in the dust.
let ’em wave their diplomas. nobody cares…
-bowerbird
I appreciate the interesting insights, but ultimately the conclusion is too foofy for me. The accuracy of the data contained in Wikipedia at any given time (for it changes constantly) will always be at the center of discussion about Wikipedia, for the simple reason that people go to Wikipedia for accurate information. If the information is only questionably accurate, or too often questionably accurate, then it stops being useful.
The argument that the *really* interesting thing about Wikipedia is that it engages people in a process of presenting information etc. etc. is revealed as unsatisfactory as soon as one ponders why that skill is useful. Answer: it is useful in the event that someone develops something like Wikipedia to use such a skill on. It’s a little self-canceling, and ultimately seems to dodge the question (much as Britannica or any number of academics would want to dodge it).
It’s a little like saying the useful thing about cars is that it teaches people about how complex machinery works. Nonsense. The useful thing about cars is that they help people move around more effectively.
Non-experts rated the credibility higher than experts, I expect, because they can only judge credibility from an article’s style, and because a very prominent stereotype and paradigm of authority in our culture is an oblique, passive-voiced, jargon-rich style that we all know and love for what it can do for us. Probably a lot of the most expert prose in Wikipedia is bored graduate students who see wiki writing as “slumming it” or as “the other side of town” and so take the opportunity to write casually and less to impress. Accuracy is an issue because it’s hard to know whether an article’s last edit introduced an error or corrected one. Articles drift ala Brownian motion and they vary widely not only with regard to how frequent the edits or “collisions” are, but also in the relative frequencies of police, vandals, cognoscienti and ignorami. (Blessed is the expert on something obscure, and cursed is the person especially expert in something we all learn as freshmen superficially and in out-dated form.) The ratio of injuries versus improvements gives an article a kind of average or equilibrium accuracy over a given slice of time–and shape its probability distribution of inaccuracies for the instant in time at which you refer to it. The wiki ideal is that an article in progress converges on a stable mature article, but they also bloat and divide, or just fail to settle down. Also the reality looks more like the ideal if you only care about accuracy and not about readability or pedagogy. It’s a lot easier for strangers to agree on facts than on pedagogy or on what needs to come first or under what heading it belongs. Wiki info doesn’t easily tend toward elegance. What articles convey they do by brute force.
It doesn’t matter how much statistical analysis and survey or representative samplings are undertaken to investigate the feasibility of wikipedia, the point is, it works for only certain things where there is a larger contemporary public who can actually relate to the material and make some assesment. Like certain pop culture. Then comes things like pharamceuticals, one can at least read up on them and then cross-index to another site on a pharmaceutical, but then, that defies the idea doesn’t it. But where it fails incredible, is at the point of engagment with history, where it has to count, in regards to our contemporary and future. Just take a small (geographically speaking) area, like Israel/Palestine – and you see what happens. The fights are ridiculous, and unless you can enjoy seeing a “neutral” heading like, how to pronounce Gaza in FIVE – thats right, at least five – Hebraic dialects… versus one arabic, and so on… it is the litmus test for how the rest of the larger epic histories are done. Think Russia, China and so on, in ENGLISH. Who knows how it is in French, Arabic, Hebrew etc… I chose english because, the war is fought from American knowledge, or lack of knowledge, bases right now. To read the wiki on those points that should help to inform us about that world we participate in, concerning historical background, is sadly failing in terms of necessary accountability. And we have learned, what is missing is not perfection in our references, but accountability. Juried references, peer reviews, fallible, but at least I know where to go.