Archive for the ‘academia’ Category
So I should start this post by first saying that I’m thrilled that Sociological Science exists. It is terrific that a group of folks did the hard work — and I imagine it’s been a lot — of putting together a high quality, open access journal that sidesteps the protracted review process we all love to hate, that evaluates quality rather than importance, and that values replication as a scientific contribution. I’ve been impressed by the caliber of the articles and love that they’re getting covered in places like Salon and Daily Kos.
In fact, it’s only because Soc Science has clearly been successful, and I think will become even more so in the future, that this is even worth bringing up: What does it mean for qualitative sociology?
At orgtheory, we’ve repeatedly discussed whether letters of recommendation (LOR) are useful or not. For some committees and tasks, LOR can help decision-making a lot – for example, undergraduate admissions, graduate school admissions, and tenure for professors. But for other committees and matters, LOR serve a per forma function and are less likely to impart information useful for decision-making. Worse, LOR can increase workloads for letter-writers, especially when they are at institutions that do not offer support for this kind of bureaucratic task.*
Nonetheless, committed letter-writers dutifully carry out their responsibilities, especially since LOR can be consequential.** As Fabio, olderwoman, David Meyer, and others have pointed out, writing effective LOR is an art, with carefully coded language that require fluency on the parts of both letter-writers and letter-readers. Otherwise, what one letter-writer might view as effusive could be interpreted as a lukewarm endorsement by readers; hyperbolic letters with hierarchical rankings ( i.e., “best student ever”) might be considered suspect. But what if LOR and related documents were brutally honest about their contexts, rather than merely following convention?
A recently published novel Dear Committee Members takes aim at these bureaucratic documents and their institutions’ crumbling support. Julie Schumacher‘s novel unfolds as a series of documents composed by a professor for various audiences. Check out the following reviews for tidbits.
Announcement from one of the hippest groups in sociology:
CALL FOR ABSTRACTS
2015 Junior Theorists Symposium
August 21, 2015
SUBMISSION DEADLINE: FEBRUARY 13, 2015
We invite submissions for extended abstracts for the 9th Junior Theorists Symposium (JTS), to be held in Chicago, IL on August 21st, 2015, the day before the annual meeting of the American Sociological Association (ASA). The JTS is a one-day conference featuring the work of up-and-coming theorists, sponsored in part by the Theory Section of the ASA. Since 2005, the conference has brought together early career-stage sociologists who engage in theoretical work.
We are pleased to announce that Patricia Hill Collins (University of Maryland), Gary Alan Fine (Northwestern University), and George Steinmetz (University of Michigan) will serve as discussants for this year’s symposium.
In addition, we are pleased to announce an after-panel on “abstraction” featuring Kieran Healy (Duke), Virag Molnar (The New School), Andrew Perrin (UNC-Chapel Hill), and Kristen Schilt (University of Chicago). The panel will examine theory-making as a process of abstraction, focusing on the particular challenge of reconciling abstract “theory” with the concrete complexities of human embodiment and the specificity of historical events.
We invite all ABD graduate students, postdocs, and assistant professors who received their PhDs from 2011 onwards to submit a three-page précis (800-1000 words). The précis should include the key theoretical contribution of the paper and a general outline of the argument. Be sure also to include (i) a paper title, (ii) author’s name, title and contact information, and (iii) three or more descriptive keywords. As in previous years, in order to encourage a wide range of submissions we do not have a pre-specified theme for the conference. Instead, papers will be grouped into sessions based on emergent themes and discussants’ areas of interest and expertise.
Please send submissions to the organizers, Hillary Angelo (New York University) and Ellis Monk (University of Chicago), at email@example.com with the phrase “JTS submission” in the subject line. The deadline is February 13, 2014. We will extend up to 12 invitations to present by March 13. Please plan to share a full paper by July 27, 2015.
Okay, so probably most of you probably aren’t at the Social Science History Association meetings in Toronto over the next few days. But some of you will be. And those lucky folks are in for four days of interdisciplinary historical social science (program here).
SSHA is my favorite conference. I’ve only been going for four or five years, but it’s quickly become the one I can’t stand to miss.
My understanding — I feel like this story used to be on the SSHA website, but I can’t find it there now — is that the organization was started in the 1970s, when quantitative history was becoming a thing. It brought together interested historians, historical demographers and economic historians, and comparative-historical sociologists into one place — basically, anyone who was interested in historical social science, or the social scientific study of history.
Over the years it’s grown and evolved, as trends have come and gone in its component disciplines. I’ve often heard SSHA described as three separate conferences that happen to meet in the same place at the same time — one of comparative-historical folks, one of historical demographers, and…I’m not sure what the third is. Economic historians? This may be true, but they all seem to get along pretty well, so far as I can tell. Maybe we’re just grateful to be somewhere where historical research doesn’t have to be justified.
However, my impression is that in the few years I’ve been going, SSHA has been attracting more and more sociologists, including a surprising number who don’t do historical work. (For example: Alice Goffman is doing an author-meets-critics panel this year. But that’s just one instance of many.)
On the one hand, it’s basically the people I like hanging out with, so awesome. On the other, I’m not sure what it means for the future of SSHA, if it becomes “Berkeley-Michigan sociology” rather than “interdisciplinary history.”
I organized a couple of panels on experts and policy (see after the jump), which I think have shaped up really well, and am presenting a paper on how the (brief) government craze for systems analysis helped spread economists throughout the federal bureaucracy in the 1960s.
I’d say drop me a line if you want to grab coffee, but you know, I’ll probably run into you.
I often wonder: why should someone blog? Philosopher John Danaher explains that it helps him:
2. It helps to generate writing flow states: I appreciate that the term “flow” state is something of a buzzword. Still, it has a basis in psychological science and it is something that blogging can help generate. The psychological barriers to writing a blog post are much lower than the psychological barriers to writing an article for peer review. Yet, when writing the former you can get into a flow state that can then be leveraged into writing the latter. Many is the time that I have finished writing a blog post and jumped straight into writing a more serious article.
Agree. Writing a blog post is like a warm up. The whole post is worth reading. The rest of the blog is fascinating as well.
Work in Progress, the blog of ASA’s organizations, occupations, and work section, just launched a new series on the future of organizational sociology. It launched today with a introduction from Liz Gorman and a first post by Howard Aldrich. Liz has an impressive slate of sociologists lined up — in the days to come, you can expect to hear from:
Martin Ruef (Duke)
Harland Prechel (Texas A&M)
Elisabeth Clemens (University of Chicago)
Ezra Zuckerman (MIT Sloan)
Gerald F. Davis (University of Michigan)
Heather Haveman (UC-Berkeley)
Brayden King (Northwestern)
Charles Perrow (Yale)
W. Richard Scott (Stanford)
Mark Suchman (Brown)
Patricia Thornton (Duke)
Marc Ventresca (Oxford)
Elizabeth Gorman (University of Virginia)
Matt Vidal (King’s College London)
Thanks to Liz and OOW for organizing this conversation and here’s hoping it gets the attention it deserves.
Long time readers know that I am a skeptic when it comes to letters of recommendation. The last time I wrote about the topic, I relied on a well cited 1993 article by Aamodt, Bryan amd Whitcomb in Public Personnel Management that reviews the literature and shows that LoR’s have very little validity. I.e., they are poor predictors of future job performance. But what if the literature has changed in the meanwhile? Maybe these earlier studies were flawed, or based on limited samples, or better research methods provide more compelling answers. So I went back and read some more recent research on the validity of LoRs. The answer? With a few exceptions, still garbage.
For example, the journal Academic Medicine published a 2014 article that analyzed LoR for three cohorts of students at a medical school. From the abstract:
Results: Four hundred thirty-seven LORs were included. Of 76 LOR characteristics, 7 were associated with graduation status (P ≤ .05), and 3 remained significant in the regression model. Being rated as “the best” among peers and having an employer or supervisor as the LOR author were associated with induction into AOA, whereas having nonpositive comments was associated with bottom of the class students.
Conclusions: LORs have limited value to admission committees, as very few LOR characteristics predict how students perform during medical school.
Translation: Almost all information in letters is useless, except the occasional negative comment (which academics strive not to say). The other exception is explicit comparison with other candidates, which is not a standard feature of many (or most?) letters in academia.
Ok, maybe this finding is limited to med students. What about other contexts? Once again, LoRs do poorly unless you torture specific data out of them. From a 2014 meta-analysis of LoR recommendation research in education from the International Journal of Selection and Assessment:
… Second, letters of recommendation are not very reliable. Research suggests that the interrater reliability of letters of recommendation is only about .40 (Baxter, et al., 1981; Mosel & Goheen, 1952, 1959; Rim, 1976). Aamodt, Bryan & Whitcomb (1993) summarized this issue pointedly when they noted, ‘The reliability problem is so severe that Baxter et al. (1981) found that there is more agreement between two recommendations written by the same person for two different applicants than there is between two people writing recommendations for the same person’ (Aamodt et al., 1993, p. 82). Third, letter readers tend to favor letters written by people they know (Nicklin & Roch, 2009), despite any evidence that this leads to superior judgments.
Despite this troubling evidence, the letter of recommendation is not only frequently used; it is consistently evaluated as being nearly as important as test scores and prior grades (Bonifazi, Crespy, & Reiker, 1997; Hines, 1986). There is a clear and gross imbalance between the importance placed on letters and the research that has actually documented their efficacy. The scope of this problem is considerable when we consider that there is a very large literature, including a number of reviews and meta-analyses on standardized tests and no such research on letters. Put another way, if letters were a new psychological test they would not come close to meeting minimum professional criteria (i.e., Standards) for use in decision making (AERA, APA, & NCME, 1999). This study is a step toward addressing this need by evaluating what is known, identifying key gaps, and providing recommendations for use and research. [Note: bolded by me.]
As with other studies, there is a small amount of information in LoRs. The authors note that “… letters do appear to provide incremental information about degree attainment, a difficult and heavily motivationally determined outcome.” That’s something, I guess, for a tool that would fail standard tests of validity.