do online journals hamper science?
James Evans (U of Chicago, Sociology) has a nice piece in the recent issue of the journal Science about how access to online journals has shaped and narrowed and perhaps hampered the way scientific knowledge is created. A very interesting piece — here’s the abstract.
Online journals promise to serve more information to more dispersed audiences and are more efficiently searched and recalled. But because they are used differently than print—scientists and scholars tend to search electronically and follow hyperlinks rather than browse or peruse—electronically available journals may portend an ironic change for science. Using a database of 34 million articles, their citations (1945 to 2005), and online availability (1998 to 2005), I show that as more journal issues came online, the articles referenced tended to be more recent, fewer journals and articles were cited, and more of those citations were to fewer journals and articles. The forced browsing of print archives may have stretched scientists and scholars to anchor findings deeply into past and present scholarship. Searching online is more efficient and following hyperlinks quickly puts researchers in touch with prevailing opinion, but this may accelerate consensus and narrow the range of findings and ideas built upon.
There’s of course also a positive side to scientific consensus and variance reduction, right? If online journal access not only “puts researchers in touch with prevailing opinion,” but also the right opinion, then presumably science progresses faster; variance reduction can be good. The acceleration of science via various technologies — print, publications, books, academic exchange, travel, conferences, the Internet — throughout history can be seen as a process of variance reduction. Wouldn’t things have progressed faster (and variance reduced) if we’d given up on ‘phlogiston’ or an earth-centric view earlier? (Though, I don’t really like to disparage historical best efforts at true facts either — at one point these represented the cutting edge of knowledge. I suppose science progresses, a la Popper, from one approximate truth to a better approximate truth.)
Now, if prevailing scientific opinion is wrong, then variance reduction of course is a problem. We’ve then got ourselves a deeper epistemological problem here about how we define knowledge: scientific consensus, logic, observation, social construction, etc. Interesting issues though!
The article (abstract, article gated). And, here’s a summary article from the most recent issue of The Economist.
The finding is really surprising to me (or it was surprising when James told me about it in the spring). You’d expect, given the rhetoric about the revolutionary potential of the web, that it would make the accumulation of knowledge more open to outsiders, etc. There should be more competition among competing paradigms, which only after a period of expansion would settle into some sort of equilibrium-like state. This suggests that exactly the opposite is occurring.
brayden
July 23, 2008 at 2:33 pm
The simplest explanation seems (unless I am missing something) to be that “true” knowledge and associated consensus is diffusing faster.
tf
July 23, 2008 at 3:39 pm
I’m not sure what you mean by “true” knowledge. Do you mean facts or do you mean theories that have been empirically verified? Because I’m not sure how opening up the knowledge pool to the internet would hasten the acquisition of either of those two things.
brayden
July 23, 2008 at 3:54 pm
Surely scientists come to a consensus about true facts much faster now, given available technologies. So, if you are toiling away on an earth-centric view, but then now all of a sudden (via some technology, or just access to information via peer-reviewed outlets) you have refuting evidence showing otherwise, you would adjust your approach accordingly — variance would be reduced, but in that case it would be a good thing. This, of course, does assume a [naive?] realist perspective and assumes that science, over time, progresses and (at least) proximately captures objective reality and true facts.
tf
July 23, 2008 at 3:59 pm
To add some nuance to this — and tease out scientific consensus effects related to “access to better information” versus “access to any information” — it’d be interesting to look at reputational effects (based on institution, journal, scientist, etc) and perhaps how the consensus develops over time. Of course the socio-political-technical machinery of it all would surely play a role (and — in this case it’d be interesting to see which journals, at what rank/’level,’ went online), but, one has to also give some credence to emergent facts and their role in developing consensus (think Watson and Crick’s Nature article — hard after that to come in with some alternative angle, consensus emerged quickly.)
tf
July 23, 2008 at 4:30 pm
Given the proliferation of competing viewpoints and explanations of events, it seems to me that the idea that increased access to information would generate increased consensus. It might generate an increase in the diversity of Kuhnian paradigms in any given arena, but really… look at the rest of society and ask yourself… does access to ‘more’ centralize or decentralize opinions on something…. I look at a grocery store and tend to think that human knowledge doesn’t work on the model of consensus building toward unified knowledge as much as consensus amongst factions, which then compete to have better an worse maps of the agreed events.
In terms of the old ‘double helix’, we have the issue of ‘when dna is in the form of a double helix, it is in the form of a double helix’ but dna doesn’t stay as a double helix nor does the double helix account for the form of dna when dna does most of its ‘work’, so does the object dna as double helix really represent what it is? or how it operates? So the double helix is consensus about one form of dna, and it established one ‘norm’ of representation of dna, but it didn’t end the debates around dna that exist today.
jeremy hunsinger
July 23, 2008 at 5:04 pm
“…it didn’t end the debates around dna that exist today.”
Jeremy: Right — and thats my Popperian hedge related to science as “approximation” rather than ultimate, final, big-t Truth. The point, however, is that progress occurs and that consensus means something. And, who emerges on top in the competition between paradigms (that you mention) and alternative views also has something to do with which paradigm has the facts right; again recognizing that power, social, political, technological issues also play a role — in the long run, though, we get better facts.
In terms of the grocery store, thats a wholly different domain from science, though yes, it’d be better if everyone converged to the true cereal: Shredded Wheat.
tf
July 23, 2008 at 5:10 pm
Dude … you are *this* close to Lyotard land with this post…
fabiorojas
July 23, 2008 at 5:56 pm
I don’t think that the grocery store is different at all. It is the same basic human endeavor, the description and organization of the world, and thus the experience of the world. I think the question of ‘who is the dominant paradigm’ is a question of organization and organizational theory, because frequently that changes depends on perspective. More information, I think generates more perspectives or minimally the possibility of more perspectives and thus paradigms, and from there I think there is less consensus toward any ‘dominant paradigm’ and more toward the paradigm of one’s current allegiance.
I’m very wary of the Saint-simonism of ‘better facts over time’ because it is a question of ‘better for whom’.
and btw, the one true cereal is seasonal, summer has always been raisin bran, winter oatmeal.
jeremy hunsinger
July 23, 2008 at 7:08 pm
Fabio: I’m going to need much more than that…I am not sure what you are after (metanarrative?).
tf
July 23, 2008 at 7:15 pm
Jeremy: Well, if we are talking about “better for whom” then we are likely to run into an impasse and epistemological incommensurability. It seems unproblematic for me to talk about progress and facts. And, I suppose it also depends on what domains we’d like to extend the paradigms-as-”knowledge” intuition — for much of the sciences the record seems rather hard to refute, in some other areas I can see your point. And, the temporal dynamic is also quite important: of course various paradigm-related factors need to be marshalled for facts or “facts” to receive consensus, but at some point things just are.
I might be on an intellectual island with all this, though I am guessing there are some other moderate realists out there as well.
tf
July 23, 2008 at 7:39 pm
Teppo: One of Lyotard’s claims in The Postmodern Condition is that we should be skeptical towards science’s authority because science is self-validating (e.g., peer review) not external truth. Evans’ finding that we’re slouching away from broad knowledge of past research toward self-validating consensus through electronic searches just gets us closer to that point. We’re not even searching through our base of knowledge. We’re just relying on the current group think.
fabiorojas
July 23, 2008 at 8:05 pm
I think it’s easier to argue about facts and true theories (i.e. laws) in the natural sciences than it is in the social sciences. There are facts in social sciences (e.g., GDP increased by 12%) but even the interpretation of those facts is highly disputed (just check out the disagreement among economists as to whether we’re in a recession now). More importantly, there are very few theories in the social sciences around which a consensus has been established. In fact, there is much more disagreement and conflict about theories in our field than there is anything approaching a consensus.
BTW, this is my major problem with evidence-based management. Until a theory of management has more than 75% agreement among the field regarding its validity, how can we unashamedly contend that there are “true” theories of management? It’s no wonder that managers are wary of just picking up journal articles and devising new practices based on their reading of them. Talk about drifting with the tide….
brayden
July 23, 2008 at 8:11 pm
Fabio:
Self-validating — I suppose, then, that I have significantly more trust in the self-validation and peer review process in terms of its check on facts and its ability to vet out matters of error. And, it’d be disappointing if Einstein et al were simply engaged in a massive within-paradigm language game. Furthermore, as noted by Strawson, I (again naively) think that scientific reasoning capabilities are in some part independent of issues of theory-ladenness.
tf
July 23, 2008 at 8:14 pm
Regarding Brayden’s comment (the first one). A Harrison White term can provide a better alternative to the market model you’ve put forth: social goo. I haven’t yet read Evans’ article, but in reading the abstract, I get the sense that the mechanism he posits — browsing through stacks, paper journals, etc. — gives one exposure to a wider range of what would seem from an efficiency perspective ‘irrelevant’ literature, some of which may end up being relevant. We might end up with cleaner sets of citiations (and clearer canons), but that messy, sticky mess is where we find new ideas.
E
July 24, 2008 at 12:24 am
Niels Jerne, the immunologist, credited Chomsky’s generative grammar for key insights in his Nobel speech.
tf
July 24, 2008 at 2:24 am
Having read the abstract (not the gated article at full length), my guess is that it is a combination of the scientist’s drive to find new resources for good ideas and bridge structural holes by creative re-combination of ideas from various sources combined with isomorphism, no matter if coercive, mimetic or normative. I could imagine that Evans’ argument can be turned into the opposite direction, as well: If Evans manages to empirically show that as more journal issues came online, the articles referenced tended to be more recent, fewer journals and articles were cited, and more of those citations were to fewer journals and articles, would that not be a sign that scientists go for the latest and most frequently cited information and sources of data? Why should they not use the global information infrastructure at their hands? Is the internet not an enabling resource both for institutions and for the various disciplines, and does the internet not foster emergent global scientific communities (of interest)? But it does question the idea of the ‘one genius’ that comes up with a completely new idea and that one great idea can be attributed to one person – rather one idea will pop up in various places, creating a different form of producing and organizing knowledge than the one we are used to, a collaborative one. And even if Evans (and other authors) decide not to publish OA, will it prevent further isomorphism? I would doubt that.
tinaguenther
July 24, 2008 at 1:15 pm