inside the american journal of sociology

Don’t worry, I won’t give away state secrets.

In the 2000-01 and 2002-03 academic years, I worked at the American Journal of Sociology as a member of the manuscript intake group and later as an associate editor. I also worked for a while, roughly at the same time, as the managing editor of Sociological Methodology, which was then edited by my dissertation advisor, Rafe Stolzenberg. In this post, I want to tell you a little bit about how top academic journals work. This is important because academics reward people based on getting into highly selective journals. There should be a lot of discussion about how the institution works and what does and does not get accepted.

Background: The AJS is the oldest general interest journal in American sociology and has, during its entire existence, been based at the Department of Sociology at the University of Chicago. To my knowledge, it has never rotated to another program. In fact, the relationship between the Department and the journal is so strong that one of Chicago’s faculty, Andy Abbott, has written a very nice monograph just about the AJS called Department and Discipline. It’s a good book and you should read it if you want to either understand the evolution of journals or how Chicago fits in to the broader discipline.

Some time ago, the AJS developed this system where students were strongly involved in the operation of the journal. For example, the AJS usually is run by a full time manager, the incredible Susan Allan, and a few students who run the office. These folks do budgets, office organization, crazy amounts of paper work, and a whole lot more. But it goes beyond administration. Students are deeply involved in the shaping the journal’s content.

At the time I was a doctoral student, the AJS was organized into three major committees: the editorial board, which is always headed by a senior professor; a manuscript intake group, which assigned reviewers to papers; and a book review board, also headed by faculty. The manuscript intake group and the book review board are mainly staffed by students. The editorial board usually has one or two students on it, who have a major voice.

In contrast, Sociological Methodology was run like many specialty journals. You had a manager (me) and the editor (Prof. Stolzenberg) who choose reviewers, read reviews, and made decisions. These two people did about 90% of the work running the journal

Lessons from working at AJS: In many ways, the AJS resembled other major journals that must process hundreds of papers per year. There is a basic intake/review/decision cycle. That process has up and down sides. The up side is that the journal review process is actually pretty decent at weeding out garbage. After a while, you can easily spot bad papers. Unending rants, poor spelling, poor formatting, lack of data. Another upside is that many papers do actually improve once people respond to reviewers.

I also saw some of the downsides of the review process. For example, I discovered that only about a third of people agree to consistently review papers, making the workload highly unequal. Some of the patterns are obvious. A lot of people stop answering the mail post-tenure. People in some sub-fields are simply bad citizens and refuse to write reviews or write bad ones. Finally, like a lot of journals, we could let papers fall between the cracks and go without a decision.

Perhaps the biggest insight that I had was the power of editors and the randomness that goes into a “great paper.” Example: while I was on the editorial board, we had a paper with ok but not great reviews. I read it and disagreed strongly. Right as the chief editor was about to assign it to the reject pile, I interjected. It was published and was covered by the national media. This may sound like a great story, which it is, it also shows the weakness of the journal system. If I had been absent that week,  or if we had another student editor, the paper would have been rejected. Conversely, I am sure that I overlooked some excellent work.

A related lesson is that the chief editor matters a great deal. An editor can doom a paper from a scholar they don’t like, or on a topic they hate, by simply assigning it to known mean reviewers. Editorial influence appears in other ways. While most papers are clear rejects, many are on the border. An interventionist editor can strongly affect what is accepted from these border line cases. One editor I worked with would actually ask the authors for the data and rerun the analysis to see if reviewer 2’s criticism was right. Another is very comfortable with adding a few suggestions and then just tossing it back to the authors. The power of editors, and the Chicago department, also manifests itself in the fact that AJS is way more tolerant of longer, theory driven papers than other peer social science journals.

A second lesson is that there are big structural factors that influence what gets published. The first factor is type of research. Simply put, ethnographers produce papers at a slower rate than demographers. So if you have a small number of papers, it doesn’t make sense to risk it all on the AJS. Instead, you move to the book or more specialized journals.That’s one reason why ethnographic work is rare in top journals  A second factor is culture. There are some sub-fields where the reviewers seem to be really difficult. For example, during the late 1990s, there seemed to be a sort of feud in social psychology. Each side would tank the others in the review process. Ethnography is similar. When people did submit field work papers, it was nearly impossible to get 2 or 3 reviewers to say “this is good enough.” Just endless and endless demands.

The final lesson I took is that we are humans and we are biased. While 95% of decisions really based on reviews, there were definitely times that our biases showed. There were one or two papers I promoted because I was excited about social movement research. At other times, decisions took into account touchy political situations and author prestige. As I said, this is not typical but it does happen and I include myself in this evaluation.

Lessons from Working at Sociological Methodology: This was a totally different experience. Instead of being embedded in a larger group, it was just literally me and a filing cabinet and my advisor. We had a weekly meeting to discuss submissions, I took notes, and he told me what to do.

Probably the biggest take home point from working with Professor Stolzenberg was that editors make or break the journal. The dude was really on top of things and few papers went past 2 or 3 months. Once a paper couldn’t get a single review after six months and the editor wrote a letter to the author explaining the situation and they mutually agreed to release the paper from review.

Stolzenberg was also not afraid of people, a strong trait for an editor. He didn’t mind rejecting people and making the process speed up. Although he didn’t desk reject often, he was good about getting reviews and writing detailed rejection letters. That way, the journal didn’t get clogged with orphaned papers. The lesson is that there really is no excuse for slow reviews. Get reviews, reject the paper, or get the hell out of the editing business.

Final note – authors and reviewers are lame: I conclude with a brief discussion of reviewers and authors. First, authors are quite lame. They are slow at responding to editor. They fail to read reviewer comments or take them seriously. And even more puzzling, they fail to resubmit after the R&R. I was shocked to discover that a fairly large fraction of AJS and SM papers at the R&R stage were not resubmitted. Perhaps a third or so. Second, reviewers are lame. As Pamela Oliver put it so well in the recent American Sociologist, the review process is simply broken. Reviewers ask for endless revisions, the focus on vague issues like framing, or simply write hostile and unhelpful reviews. So I thank the 1/3 of academics who write prompt and professional reviews and I curse the 2/3 of shirkers and complainers to an eternity of reading late reviews that criticize the framing of the paper.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street