comment on comments
The conversation started by Jeremy and continued by Brayden raises interesting questions about journal article comments. I just had to put my $.02 on the whole topic.
-
It seems that comments serve an important function, analagous to book reviews for academic monographs. Without them, articles would stand “uncontested” within the literature, unless someone where to invest serious time writing something that could pass muster as a stand alone article. For issues such as pointing out technical errors, extra analyses that cast a different light on the argument, or alternative readings of the same data, full length articles aren’t appropriate, though the comment would improve the state of knowledge. The comment seems really appropriate as a form of brief, but useful, professional communication. Also, future scholars can learn about the state of debate around the time of the article.
-
Though I don’t have a cite for it, my impression is that comments are relatively rare compared to a few decades ago. I am not sure why this is. Perhaps comments are now so devalued that scholars can’t be bothered. The drive for full length articles for hiring and promotion might have had the effect of suppressing serious commentary of published work. If so, it’s a shame to have created incentives that suppress criticism.
-
Much of the comments to Jeremey’s original thread touched on making enemies via comments. If comments were mean spirited, it’s ok to be mad, but don’t we have room for honest intellectual disputes in the form of professional comments? If we can’t openly critique ourselves, doesn’t that send the signal that sociology, or whatever discipline, is not open to real criticism? Doesn’t a “no comment zone” say that we’re not a community of skeptical inquiry?
Comments, of course, are welcome.
orgtheory.net 
However rare comments may be, I’m not sure what those that do appear in sociology are submitted comments. A nontrivial portion are comments that appear are invited by the editor somewhere along the publication process, such as because the person invited to comment was a reviewer. Submitted comments are pretty irrational things to do on the face of it: they take time to do, and if the journal doesn’t go for it, it can be hard to figure out what to do with it.
Myself, I wish there was some kind of forum that was not quite a journal, but more than just sticking something up on your website, for things like comments, or brief analytic notes on an article, or unsolicited book reviews, etc.. This is part of my more general feeling that there is not enough direct interaction about work, and especially not over any kind of reasonable timescale.
LikeLike
jeremy
December 26, 2007 at 5:36 am
There is no such forum in sociology that I know of, aside from blogs. Can blogs be a place for serious comments on articles?
LikeLike
Fabio Rojas
December 26, 2007 at 6:22 am
This forum could be called “Sociological Letters”, emulating the “Physics Letters” series.
To me, the problem is that sociology is not written in a way that encourages comments or replication. For example, I find it regrettable that people do not comment more often on one’s counterfactuals (so many articles simply affirm the consequent that this check is usually very good).
By the way, I know at least one journal which almost systematically publishes at least one comment with each article (French journal Sciences Sociales et Santé).
LikeLike
Fr.
December 26, 2007 at 6:38 am
“Can blogs be a place for serious comments on articles?” — Of course they can be, but should they be?
Other suggestion, if preprints had a bigger role in publication, comments may flow more easily (witness arxiv). But again, there is something structural to sociology that acts as a disincentive to commenting and/or replicating.
LikeLike
Fr.
December 26, 2007 at 6:40 am
It does seem that electronic journals will/would make all these issues better, as they would naturally permit cross-linking and discussion. The only real problem is to deal with the status issues for tenure cases. Once we solve that, it seems that knowledge would be enhanced.
LikeLike
olderwoman
December 26, 2007 at 3:35 pm
I agree that it is good for the discipline(s) to have free communication and active debate between scholars and that the journal comment often serves this function, but I think it has to be one of the most inefficient forms of debate that we have. It takes forever to get a journal article in print once the study is actually completed. Comments don’t take quite as long to get through the publication process but my sense is that most comments that are submitted never get published. Adding months between the printing of an article and the publication of a comment seems to have a stifling effect on the debate/conversation. A better outlet would quickly publish comments and allow more free-flowing interaction. A blog may be one possible outlet for this type of debate, but given the costs of imposing this technology on everyone (as a commenter once noted here at orgtheory), it might be better to have an active collective forum where people could posts comments. The market of public attention could decide which comments were worthy of notice and which should slip through the cracks.
LikeLike
brayden
December 26, 2007 at 4:45 pm
Fr & Olderwoman: I like the concept of “soc letters,” and maybe it’s a good project for a senior faculty member. Also, when it comes to promotion, I’d treat most comments like book reviews. Valuable, but really not on the same league as articles & books, unless the individual person wants to make a case that a specific comment rises to a higher level. The assumed “tenure value” of a comment should be nil, unless there’s a strong obvious case otherwise.
Brayden: I think there is an easy solution – have the editor and editorial board expedite reviews for comments. Many specialty journals actually have pretty good turn arounds by working through an editorial board. Maybe an author could even submit a short 1 paragraph summary, for screening. We could also limit page length to help speed things up. We could also add “online first,” to help speed things up. Yes, current publishing is inefficient, but it doesn’t have to be.
LikeLike
fabiorojas
December 26, 2007 at 7:19 pm
In the back of my mind is the belief that comments almost always have an element of social status ordering vis-a-vis the field that are played out among elites in top journals. I’m thinking about Jeremy’s tone in his original post (“S/he also does something in the paper that I personally regard as disingenuous, or ignorant to the point of negligence (I’ll let you choose which of these is less harsh)”), and also the whole tone around the Duneier-Klinenberg pieces – though a research note, it fits the same genre.
Don’t you think the unspoken thing here is that sociologists don’t do incremental science very well, which would be great; but instead tend to do status? Or is that totally ungracious?
LikeLike
Peter
December 26, 2007 at 7:36 pm
A journal article is nothing more or less than a permanent record (a totem if you like) of a contribution to knowledge. In the case of a major journal it signals a “significant” contribution to knowledge. I think it highly appropriate that comments to be used to signal controversial contributions as such. For instance, the highly cited Priem and Butler article in AMR on the resource-based view attracted a good number of comments that delineated various positions on their controversial argument. If this makes editors more likely to publish controversial arguments then all the better.
LikeLike
stevphel
December 26, 2007 at 8:24 pm
Peter – I agree with you about the status aspect of comments. That’s one reason it might be a good idea to move comments out of the journals and into online forums. If you made it easier for people to make comments, I think you’d see more democratic participation and we could judge comments based on their worth rather than on the status of the people making them. Another reason that increasing the space for comments (i.e. online comments) is a good idea is that it would make it possible to publish comments that make small but important corrections to a published article. Right now the bar for publishing a comment may be too high, which means that a lot of comments that point to minor problems in articles are never broadcast.
Fabio – I like the idea of expediting the review process, but why not make the entire comment section “online only.” You’d increase the available space for comments and you’d have these other advantages.
LikeLike
brayden
December 26, 2007 at 9:18 pm
Brayden: An complete online peer reviewed comments sounds good to me. I can imagine situations where maybe editors want to “convert” online comments into print forums if the level of discussion merits it.
LikeLike
fabiorojas
December 26, 2007 at 10:30 pm
I do think it’s an interesting question what motivates people to write comments, since otherwise there are plain incentives against it. I don’t think there is a “doing status” thing motivating the comment I wrote about in my post, but then again it’s not like I am a big believer that people have that great of insight into their motivations. I think a theme that can be drawn from several of my pieces is that I have a crusaderly streak. One such crusade is about the wild lack of critical thinking that many sociologists evince toward work that claims to show the triumph of some sociology-affirming narrative against some outsider (esp. “biological”, but also psychology or economic) alternative.
LikeLike
jeremy
December 27, 2007 at 9:36 pm
wild lack of critical thinking that many sociologists evince toward work that claims to show the triumph of some sociology-affirming narrative against some outsider.
Jeremy, I think that’s an insightful thing to say (and here I go a bit off topic). But it sounds so, I don’t know, unreasonable. Why should we give the benefit of the doubt to economic, psych, or biological explanations over sociological ones? Do you imagine economists giving the benefit of the doubt to outsider narratives over market ones? Or psychologists? Or biologists? Really?
I once was at one of those over-and-over sessions on economists-meet-sociologists-meet-economists, and the economist on the panel said he has from time to time read good sociology – in fact there was this really neat study of life insurance and cultural changes in the US!! Of course, he didn’t remember who it was or what the details were, despite the fact that Viviana Zelizer was sitting next to him at the panel. My point is that the baseline in every discpline is to favor one’s own explanations and to be skeptical towards others. Why discount the sociology-affirming narrative so easily?
I know that in an ideal scientific community there would be a deep openness towards other perspectives in the interests of scientific truth. But of course we don’t live in an ideal world, but a scholarly-partisan one (this was why I was thinking that we fall back on more status-ordering games in my previous comment).
LikeLike
Peter
December 27, 2007 at 11:19 pm
I agree with Jeremy. In sociological and organizational outlets there at times are hopeless ideological “love fests” and self-congratulatory narratives which would be significantly enriched by some critical commentary from neighboring disciplines – psychology, economics, etc. (Vice versa of course is also true.)
Let the truth take us wherever it may.
LikeLike
tf
December 27, 2007 at 11:28 pm
Fabiorojas (12-26): I actually think the journals ought to be online too, not just the comments. Hence my concern about the tenure process. On-line articles with comments enabled would seem a good thing. Might also facilitate the out-of-paradigm critiques.
LikeLike
olderwoman
December 28, 2007 at 12:32 am
A few notes on this very interesting discussion:
a. It would be nice to know why commentary and debate declined so much over the past twenty-five years, at least in ASR. In the 1950s and 60s, ASR had a regular “communications” or “communications and opinion,” which included one or two short ‘letters to the editor’ commenting on papers, with replies from the authors. And a short and very rough JSTOR check (by looking at how many articles had ‘comment in the title’; this seems to exclude any non-comments but misses some comments, including one that I wrote) shows that even after this section was eliminated, there were many more comments than we now see– from 1970 through 1984, I count 82 comments in ASR; from 1985 through 2004, there were only 25. The trend seems similar at AJS, but with significant editor-dependence. There was still a lot of commentary and debate through 1991, when Tienda took over for Parish. For 1992-1996 period, there were only three comments/replies in total, whereas the previous five years had 54. There was a bit of a reversal in the 1997-2001 period (under Laumann and Gould), but then only three since (largely under Abbott). (I could find no note by Tienda explaning the change in policy. It would be interesting to ask her.)
b. I agree that it would be great to have much more commentary and debate in public. An online journal that includes commentary and debate on articles in flagship soc journals is a very interesting idea. I say this despite the fact that I was burned very badly by editorial decisions in the handling of a comment/debate a few years ago (which allowed for changes in the original paper that then rendered false some things I wrote in the comment). I actually think that my bad experience partly reflects what happens when these debates are so rare. If they were a normal part of the publication process, editors and other parties would understand their roles better and the level of tension would diminish. And perhaps a cross-journal forum that was expressly focused on commentary and debate might further normalize it.
c. I have to side with Jeremy (and TF) on his crusade against the “wild lack of critical thinking that many sociologists evince toward work that claims to show the triumph of some sociology-affirming narrative against some outsider (esp. “biological”, but also psychology or economic) alternative.” This was exactly my motivation for writing the comment I referred to above (in the June 2004 issue of ASR). I guess my response to Peter is twofold. First, ‘two wrongs do not make a right’. Sure, economists can be incredibly frustrating in their tendency to ignore sociology, to use sociological ideas or methods without attribution, etc. But I just don’t see a principle other than competitive self-interest that might justify our following a similar attitude towards economics. And I actually think it’s in our competitive self-interest to turn the other cheek. It’s not like we’re offering them jobs in sociology departments or something. It’s just a matter of taking the time to read, understand, and use relevant work. Second, the problem can be much worse than simply ignoring relevant work. The nub of the problem is when a sociologist motivates his/her work by attacking a caricature of work in another discipline. (The targeted disciplines are typically not just ‘other’, but higher-status than sociology). All too often, reviewers don’t know enough to tell that a straw man is being attacked and they are not motivated to find out what the real story is, as they are too willing to have their prejudices confirmed.
LikeLike
ezrazuckerman
December 28, 2007 at 5:02 am
[…] and proceed as if the other disciplines did not exist. Consider the following comment from Peter to Wednesday’s post: wild lack of critical thinking that many sociologists evince toward work that claims to show the […]
LikeLike
we don’t need no other stinkin’ disciplines « orgtheory.net
December 28, 2007 at 5:05 am
Ezra: Yours is not the first case I have heard of “comment burn” caused by authors retroactively changing articles after seeing comments. Perhaps the best policy is to permit submission of comments after an article has been printed. It’s an easy and obvious solution. This might be another reason that comments are on the decline – who wants to have their words appear non-sensical after the original article has been changed?
LikeLike
fabiorojas
December 28, 2007 at 5:18 am
Yes, apparently authors often want to change their papers once they have seen the comment (even when they are given the opportunity to reply). In fact, the editor who solicited my comment warned me that authors try to do this. The problem was that there was an editorial transition and the new editor was… shall we say… less concerned with this issue.
LikeLike
ezrazuckerman
December 28, 2007 at 5:51 am
As a follow up to Peter’s point I’ve written a post on Scatterplot.
LikeLike
jeremy
December 28, 2007 at 8:46 pm
Econ journal watch is a good model of Brayden’s idea. I wouldn’t mind having a Soc journal watch.
LikeLike
Omar
January 13, 2008 at 12:46 pm