Archive for July, 2007

Privileged Peer Review – whose opinion counts?

Tuesday, July 24th, 2007

While in Lisbon earlier this month, I was approached by a Masters student who expressed skepticism expressed by her review panel on connectivism. Our conversation was short, and the situation likely more complex than expressed during the brief exchange, but she stated that her panel felt that the theory of connectivism and the book Knowing Knowledge had not been subject to peer review.
I’ve heard this complaint several times – generally from students in Masters or PhD level programs. While I sympathize with the challenges of these students, I’m not entirely inclined to pursue (exclusively) publishing in peer review journals (though I have succumbed somewhat recently, with several articles in review – should be out in 2008 or so :) ). Peer review plays an important role – it is intended to provide expert critical review of concepts and ideas to ensure quality and accuracy. I’m all for that. My primary concern rests with “privilege only” accepted view of peer review. The progressive advancement of educational attainment (see OECD’s Education at a Glance 2006) indicates a society increasingly capable of engaging in complex dialogue. The throne of knowledge is now a seat available to many of society’s members. As such, it’s reasonable to assume that the opinions of even those peripherally engaged in a discipline can provide insight and value. I appreciate experts, excellence, and established processes. But I despair when the processes of validation inhibit, rather than advance, thinking and idea sharing in a discipline.
A few additional concerns:
1) Peer review is by no means a singular process of determining and ensuring quality. Implementation and context of use ultimately determine the viability of ideas. Einstein’s famous thought experiments created an impact by opening conceptual. Ultimately, even after peer review, the ideas were tested in the real situations and, over a period of decades, were amended, disproved, or upheld. Often, support came from related fields – initial analysis of Einstein’s general theory of relativity resulted in acceptance from peers in that it addressed (among other things) the problem of perihelion shift of Mercury. Experimental support was not forthcoming until much later.
2) Peer review is based on the assumption that a handful of experts are capable of validating ideas. In reality, even experts get it wrong. We all do sometimes. Opening the review process to the broadest possible number of responses, increases the possibility of quality evaluations, even though the increased commentary may appear to cloud the situation with clutter. Good ideas, however, have a way of getting connected and noticed. See swarm theory for how the aggregated activities of many singular agents can address astonishingly complex problems. Some sites, like Nature have opted to embrace open peer review. In a previous article – Scholarship in an Age of Participation – I’ve called for a model of scholarship which honors expert peer review, yet opens the door to greater involvement from the larger community.
3) Peer review time lines in traditional journals can be lengthy – months, even years. By the time the article hits the pages of the journal, the research may be outdated. While some fields develop more slowly than those impacted by technology, educational technology in particular requires a shorter lead time. Instead of relying solely on expert validation, why not permit the minds of many to grind the edges from initially posited ideas.
4) The high-privilege of the author. Authors are like lecturers – one way broadcast of information. This can be quite delightful. But the two-way flow of many developments online can be a much more rewarding experience for both the author and the audience. PloSONE allows for collaborative review and annotation (though glancing through most articles, these features are not used extensively). When writer and audience, author and reviewer, are engaged in dialogue, more fully developed ideas and more integrated concepts are the reasonable outcome.
I did a quick search on connectivism. First page of results are as to be expected: elearnspace, connectivism.ca, downes.ca, LTC online connectivism conference, etc. By the second page (results may vary based on which country location the search is conducted), we encounter Plon Verhagen’s review of connectivism. He didn’t like it. By page 4, Bill Kerr expresses his concerns with the theory. Continue on with the search, and you’ll encounter numerous expressions of support, uncertainty, and disagreement. The reactions come from teachers, professors, researchers, business people, theorists, conference proceedings and so on. Is that peer review? Or is peer review only what happens when it is under a privileged process?
The problem with peer review does not rest with connectivism (or any other such concept expressed in non-traditional means). Connectivism has been extensively peer reviewed. I would suggest much more so than most academic articles. In the process of review, others have built on, extended, and formed the theoretical basis of connectivism (see Stephen Downes’ paper on Connective Knowledge for an epistemological basis of connectivism. He may not have intended it as such, but I see certainly many points of compatibility…and a few for debate :) ).
The real problem rests with any closed method that values a singular process too highly. Take 10 minutes. Use Google (or if particularly concerned, try Google Scholar). Research the ideas expressed. Search the authors of critiques and associated articles. The world has changed. The devices (books, paper journals) and processes (closed peer review) have given way to an overwhelming array of options. A grad student has a perspective worth considering. So does someone from related fields. In my view: throw open the doors which permit dialogue. Move foundational elements of scholarship into the hands of everyone who chooses to be engaged (”citizen scholars” as stated in this report (.pdf) by the Council of Graduate Schools. Insular evaluations are not as effective as commentary from numerous perspectives. The experts still have a role (obviously). But not as privileged as previously. In this mode, an expert carries authority in dialogue based on continual contributions and reputation. Peer review, in its current conception, appears to be more concerned with preserving privilege than with advancing knowledge. I find more zeal, energy, and creativity in the chaotic edublog community than I find in formal journals. And I personally would rather stand with others on the fertile verge of technology and education defined by enthusiasm and a spirit of advancement than to pursue a method of knowledge validation antiquated for our scholarly needs today.
UPDATE: As luck would have it, I just came across two papers on peer review and academic publishing in Inside HigherEd Articles: Ideas to Shake Up Publishing and Is Peer Review in Peril?

We are not neutral

Tuesday, July 17th, 2007

Earlier this week, I received a link (from Peter Tittenberger) to a CBC series titled Spin, the spinners and the spun. The audio series (about 6 hours in total) chronicles the rise of public relations in media. What we now see as commonplace – press releases, sound bites – was initially distrusted by media (which at that point was basically the newspaper industry). The press kept its distance from public relations of business, politics, and others with a particular message. The press was, essentially, a neutral player in public discourse, playing a role of informing the public of notable events.
Toward the end of the 19th century, Ivy Lee pioneered the perception of press releases as news. Edward Bernays was not far behind Lee, though he quickly moved beyond seeing public relations as a means to provide the press with news, and instead sought to manipulate and influence others through the medium. Bernays notion of engineering consent was an elaborate foray into directing the actions of others (crowd psychology) to his intended aims. Consider his “torches of freedom” campaign on behalf of American Tobacco to encourage women to smoke. The deliberate shaping of public opinion was central in Bernays’ interactions with the press. Bernays’ and Lee’s principles formed the basis for how corporations and politicians view media – as a megaphone for presenting their message.
The press and news industry began a somewhat tortured relationship with politicians, corporations, and special interest groups. For someone with a message, “the news” was a vehicle for entering the minds of citizens. Carefully crafted press releases and media “talking points” for politicians have infiltrated the press to the point where authentic voices are obscured with hidden marketing messages.
Why this discussion on this blog? I’ve followed criticism leveled at blogs, wikis, “wisdom of the crowds”, and other views of “the masses”. Generally, the tone is one that the activities of the many fails to meet the quality of the experts or the few. This may well be the case in some situations, but certainly not all. Even more important, especially when applied to news media like radio, TV, magazines, and newspapers, is the lack of neutrality. While an expert reporter may provide important investigative coverage, she is not the norm. Have a look at the Center for Media and Democracy’s report on Still not the News, covering how media outlets use press releases provided by corporations/government. These releases are sometimes reproduced in their entirety without any acknowledgment that they are actually advertisements. Pharmaceutical companies do this frequently…and in one instance, different news programs simply narrated (word for word) over the press release provided by the company. The fault here is not with the companies. It’s with the news agencies that fail to acknowledge the lack of neutrality of the sources and with the public that esteems them too highly.
If amateurs lack the ability of experts, then it’s important to acknowledge that the experts in different media are not neutral. They are often mouth pieces for the PR spin of others. While the authenticity of citizen journalism is being questioned, we should be questioning the lack of disclosure by traditional channels. As this wiki article states: “Citizen Journalism is slowly being looked upon as a form of rightful democratic ways of giving honest news, articles, etc, directly by citizens of the world from anywhere.”
Our distraction with addressing the concerns of critics has been misplaced. For example, wikipedia is challenged as being inaccurate (though an “expert-led investigation” by Nature suggests inaccuracies between Wikipedia and Britannica are not that significant). Wikipedia, however, says more about information access than it does about authority. Similarly, citizen journalism says more about openness and transparency than it does about professionalism. There is room enough for both expert and amateur, Wikipedia and Britannica, bloggers and reporters.
Where does this leave us in relation to education? We are not as far down the road of change in education as news, PR, and other media. But we are seeing some distinctions emerging. The read/write web threatens a significant change in how we interact with information and each other. Mobile devices allow students to ignore the restrictions of network administrators. As educators, we still have a stronghold on accreditation. But what happens if accreditation is no longer the requirement of learners? What happens if the capacity to perform a task (such as programming with Ruby on Rails) is more about reputation within a field than a piece of paper provided by an institution. (I should lament, however, that I have issues with the utilitarian focus of learning – i.e. learning only for a career or a job. Learning has deeper elements to it that simply work-based – it prepares us for citizenship, for life…and, I think, provides a sense of enjoyment not unlike eating exquisite food).
Much like Britannica fails to see the draw of information access, or newspapers fail to see the draw of authentic voices, I’m concerned that educators are not seeing the draw of learning and personal control. When an industry provides its audience/customer with a service or product no longer desired, it risks obsolescence. Obviously educational institutions are far from collapsing, though some, like Graham Atwell, question whether it is the beginning of the end of industrial schooling (overlooked in the discussion is the partnership with Microsoft – close public schools open corporate schools). But all of our philosophies and high ideals bear little weight when the industry we serve has shifted.
I’ve been thinking about the role of universities in society – I don’t think we should treat the so-called web 2.0 as something to which we must automatically move simply because students are adopting these tools. The tools need to make sense in the context of education. Education, after all, does not play only a reactionary role in society, but also a role of transformation. Yet in our emphasis on the “how” of education, we have lost sight (as Postman states) on the “why” of education. I think many students have a different version of “why” than most educators. This is the heart of our challenge (opportunity). It’s not about finding better tools and approaches to teaching. It’s about why we do it at all. Ask the news industry what happens when your audience has a different “why” than you do…