orgtheory.net

Search Results

students evaluations are garbage and so are letters of recommendation – but NOT gre scores, haters!

For a long time, I believed that student evaluations were valid measures of teaching effectiveness. My belief was based on the following issues.

  • First, there are a fair number of studies that claim a correlation between student evaluations and learning. The critics conveniently overlook this literature.
  • Second, I believed that students can spot a miserable teacher. You don’t need to be steeped in pedagogy theory to see if an instructor is disorganized, or is simply a horrid lecturer.
  • Third, most complaints about student evaluations seemed pretty self-interested. Who complains about evals the most? The professors!* Doesn’t mean they are wrong, but one should examine self-interested claims with some caution.
  • Fourth, critiques of student evaluations of faculty are often couched in bad logic. For example, if an instrument is biased against group X, it doesn’t mean automatically that the instrument is not consistent or valid. It might be the case that the instrument is less valid and consistent for group X, but still points in the right direction. You can only say that student evaluations are “worthless” if the correlation between evals and learning is zero and that is a stubbornly empirical point. Yet, critics in the popular media jump from bias to a lack of validity.

But over time, there have been a parade of better studies that explore the link between outcomes and evaluations and the answer is often null. So what should any seriously interested person do? Wrong answer: Cherry pick studies that confirm one’s belief. Better answer: look for a meta-study that combines data from new and old studies. This fall, Studies in Educational Evaluation published on such meta-study of student evaluations of teacher. Bob Uttl, Carmela White, and Daniela Wong Gonzalez performed such a meta-analysis can come to the following conclusions:

• Students do not learn more from professors with higher student evaluation of teaching (SET) ratings.

• Previus meta-analyses of SET/learning correlations in multisection studies are not interprettable.

• Re-analyses of previous meta-analyses of multisection studies indicate that SET ratings explain at most 1% of variability in measures of student learning.

• New meta-analyses of multisection studies show that SET ratings are unrelated to student learning.

There article is not perfect, but it is enough to make me seriously reconsider my long standing belief in student evaluations. I am very willing to consider that student evaluations are garbage.

However, I want to the reader to be consistent in their intellectual practice. If you believe that student evaluations are bunk, then similar evidence suggests that letters of recommendation are garbage as well. Here is what I wrote two years ago:

I slowly realized that there are researchers in psychology, education and management dedicated to studying employment practices. Surely, if we demanded all these letters and we tolerated all these poor LoR practices, then surely there must be research showing the system works.

Wrong. With a few exceptions, LoRs are poor instruments for measuring future performance. Details are here, but here’s the summary: As early as 1962, researchers realized LoRs don’t predict performance. Then, in 1993, Aamondt, Bryan and Whitcomb show that LoRs work – but only if they are written in specific ways. The more recent literature refines this – medical school letters don’t predict performance unless the writer mentions very specific things; letter writers aren’t even reliable – their evaluations are all over the place; and even in educational settings, letters seem to have a very small correlation with a *few* outcomes. Also, recent research suggests that LoRs seem to biased against women in that writers are less likely to use “standout language” for women.

The summary from one researcher in the field: “Put another way, if letters were a new psychological test they would not come close to meeting minimum professional criteria (i.e., Standards) for use in decision making (AERA, APA, & NCME, 1999).”

 

If you are the type of person who thinks student evaluations are lousy, then you should also think letters of recommendation are garbage as well. To believe otherwise is simply inconsistency, as the evidence is similar in both cases.

While I am at it, I also want to remind readers that similar analysis shows that standardized tests are actually not bad. When you read the literature on standardized tests, like the GRE, you find that standardized tests and grades are actually correlated – the intended purpose. And I haven’t seen many other meta-analyses that over turn the point.

To summarize: student evaluations and letters of recommendation are bunk, but standardized tests are not.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome!

*For the record, my evals range from slightly below average to very good. And I’ve actually won multiple teaching awards. So this is not a “sour grapes” issue for me.

 

Advertisements

Written by fabiorojas

January 5, 2018 at 4:53 am

yes, even mediocre students deserve letters of recommendation

Yes, I believe that letters of recommendation are garbage. But if we continue to require letters, faculty have a moral obligation to write them. Why? Part of being an educator is to evaluate students for the public and as long as they subsidize us professors, we need to satisfy the external demand for assessment.

Sadly, many professors take an opposite view. Students often report that professors turn them down. That happened to me all the time in graduate school. Letters were a precious commodity reserved for the best students. That is simply wrong.  In a great post at Scatter, Older Woman explains why you should write letters for most students:

The combination of a high workload per student who needs references and claims that all letters should be excellent or not written at all leads many instructors to refuse to write letters for any but A students or students they know well.  But is this fair?

Her answer?

There are a lot of graduate and professional programs out there with widely varying degrees of selectivity. Virtually all of them require three letters of reference for an application to be complete. Getting those three letters is a nightmare for some students because they have trouble tracking down their past instructors and some they do track down refuse to write for them for reasons ranging from the student’s mediocrity to the instructor’s sabbatical or general busyness. I have had conversations in which I tell a student that the letter I could write for them would not be a very good letter and the student would say: I don’t care what it says, I just need three letters. I’ve also talked to honors students who have done independent projects and have one or two excellent letters nailed down who are still desperately shopping for somebody, anybody, to write their third letter, because no matter how good the first two letters are, the application will not be complete without the third.

My view is that all of us who are regular faculty (either tenure track or non-contingent adjuncts) should treat writing letters of reference as an often-annoying but important part of our job. These letters should be honest, and we certainly owe it to the student to tell them honestly if the letter we would be able to write would be tepid or contain negative information that would not help them. We also owe it to the student to ask them about their plans, about their perceptions of the selectivity of the program they are applying to, and whether they have done their homework in selecting a program that fits their qualifications. But if the student feels they want or need the letter anyway after this disclosure and discussion, we should write the letter.

Correct! Basically, letters are not the special property of A students. Many graduate programs simply want to know that the person did decently. Instructors are not required to write special letters for everyone. Most students just want a few sentences explaining that they showed up and did relatively decently. In fact, I think it is totally ok to write one form letter for decent, but not great, students that you can customize as you see fit. It is a requirement for large, public institutions.

Heck, you can even write short and honest letters for crummy students. A real example: In my first year teaching, a dude name Jiffy* asked me for a letter. He was a really weak student. C in intro sociology and seemed spaced out. I said, “sure, but the letter will reflect your current grade – C.” He said that was totally ok. All he wanted was a study abroad letter and all it needed to say was that he attended class and was passing. And so I wrote that letter. All I wrote was a paragraph saying that he showed up to class and would answer questions if called upon. That’s it.

I never did hear back from Jiffy but I Googled him a year ago. He’s now a successful dentist. And you know what, if I helped some dentist enjoy a semester abroad, that’s not a bad thing.

Bottom line: Quit your whining and write that letter. If you don’t think it is part of the job, get another job.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome!

*Not a real name.

Written by fabiorojas

November 16, 2017 at 5:08 am

Posted in academia, fabio, teaching

my deep burning hatred of letters of recommendation

Econjeff mentions my long standing critique of letters of recommendation (LoRs). Here, I describe my personal experience with them and then I restate the massive empirical research showing that LoRs are worthless.

Personal experience: In graduate school, I had enormous difficulty extracting three letters from faculty. For example, during my first year, when I was unfunded, I asked an instructor, who was very well known in sociology, for a letter. He flat out refused and told me that he didn’t think I’d succeed in this profession. In the middle of graduate school, I applied for an external fellowship and was informed by the institution that my third letter was missing. Repeatedly, I was told, “I will do it.” Never happened. Even on the job market, I had to go with only two letters. A third professor (different than the first two cases) simply refused to do it. Luckily, a sympathetic professor in another program wrote my third letter so I could be employed. Then, oddly, that recalcitrant member submitted a letter after I had gotten my job.

At that point, I had assumed that I was some sort of defective graduate student. Maybe I was just making people upset so they refused to write letters. When I was on the job, I realized that lots and lots of faculty never submit letters. During job searches at Indiana, I saw lots of files with missing letters, perhaps a third were missing at least one letter. Some were missing all letters. It was clear to me that l was not alone. Lots of faculty simply failed to complete their task of evaluating students due to incompetence, malice, or cowardice.

Research: As I grew older, I slowly realized that there are researchers in psychology, education and management dedicated to studying employment practices. Surely, if we demanded all these letters and we tolerated all these poor LoR practices, then surely there must be research showing the system works.

Wrong. With a few exceptions, LoRs are poor instruments for measuring future performance. Details are here, but here’s the summary: As early as 1962, researchers realized LoRs don’t predict performance. Then, in 1993, Aamondt, Bryan and Whitcomb show that LoRs work – but only if they are written in specific ways. The more recent literature refines this – medical school letters don’t predict performance unless the writer mentions very specific things; letter writers aren’t even reliable – their evaluations are all over the place; and even in educational settings, letters seem to have a very small correlation with a *few* outcomes. Also, recent research suggests that LoRs seem to biased against women in that writers are less likely to use “standout language” for women.

The summary from one researcher in the field: “Put another way, if letters were a new psychological test they would not come close to meeting minimum professional criteria (i.e., Standards) for use in decision making (AERA, APA, & NCME, 1999).”

The bottom line is this: Letters are unreliable (they vary too much in their measurements). They draw attention to the wrong things (people judge the status of the letter writer). They rarely focus on the few items that do predict performance (like explicit comparison). They have low correlations with performance and they used codes that bias against women.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

October 6, 2015 at 12:01 am

letters of recommendation: still garbage

Long time readers know that I am a skeptic when it comes to letters of recommendation. The last time I wrote about the topic, I relied on a well cited 1993 article by Aamodt, Bryan amd Whitcomb in Public Personnel Management that reviews the literature and shows that LoR’s have very little validity. I.e., they are poor predictors of future job performance. But what if the literature has changed in the meanwhile? Maybe these earlier studies were flawed, or based on limited samples, or better research methods provide more compelling answers. So I went back and read some more recent research on the validity of LoRs. The answer? With a few exceptions, still garbage.

For example, the journal Academic Medicine published a 2014 article that analyzed LoR for three cohorts of students at a medical school. From the abstract:

Results: Four hundred thirty-seven LORs were included. Of 76 LOR characteristics, 7 were associated with graduation status (P ≤ .05), and 3 remained significant in the regression model. Being rated as “the best” among peers and having an employer or supervisor as the LOR author were associated with induction into AOA, whereas having nonpositive comments was associated with bottom of the class students.

Conclusions: LORs have limited value to admission committees, as very few LOR characteristics predict how students perform during medical school.

Translation: Almost all information in letters is useless, except the occasional negative comment (which academics strive not to say). The other exception is explicit comparison with other candidates, which is not a standard feature of many (or most?) letters in academia.

Ok, maybe this finding is limited to med students. What about other contexts? Once again, LoRs do poorly unless you torture specific data out of them. From a 2014 meta-analysis of LoR recommendation research in education from the International Journal of Selection and Assessment:

… Second, letters of recommendation are not very reliable. Research suggests that the interrater reliability of letters of recommendation is only about .40 (Baxter, et al., 1981; Mosel & Goheen, 1952, 1959; Rim, 1976). Aamodt, Bryan & Whitcomb (1993) summarized this issue pointedly when they noted, ‘The reliability problem is so severe that Baxter et al. (1981) found that there is more agreement between two recommendations written by the same person for two different applicants than there is between two people writing recommendations for the same person’ (Aamodt et al., 1993, p. 82). Third, letter readers tend to favor letters written by people they know (Nicklin & Roch, 2009), despite any evidence that this leads to superior judgments.

Despite this troubling evidence, the letter of recommendation is not only frequently used; it is consistently evaluated as being nearly as important as test scores and prior grades (Bonifazi, Crespy, & Reiker, 1997; Hines, 1986). There is a clear and gross imbalance between the importance placed on letters and the research that has actually documented their efficacy. The scope of this problem is considerable when we consider that there is a very large literature, including a number of reviews and meta-analyses on standardized tests and no such research on letters. Put another way, if letters were a new psychological test they would not come close to meeting minimum professional criteria (i.e., Standards) for use in decision making (AERA, APA, & NCME, 1999). This study is a step toward addressing this need by evaluating what is known, identifying key gaps, and providing recommendations for use and research. [Note: bolded by me.]

As with other studies, there is a small amount of information in LoRs. The authors note that “… letters do appear to provide incremental information about degree attainment, a difficult and heavily motivationally determined outcome.” That’s something, I guess, for a tool that would fail standard tests of validity.

50+ chapters of grad skool advice goodness: Grad Skool Rulz/From Black Power 

Written by fabiorojas

October 29, 2014 at 12:01 am

what is the secret code for letters of recommendation?

On this blog I’ve been told to avoid certain things in letters of recommendation. For example, “hard working” has a bizarre secret reading of “untalented.” Someone once told me that “please call me,” a boiler plate sentence, actually means “this person is psychotic but I can only tell you over the phone.” Allegedly, saying that one is good at teaching, has good people skills, or is caring also broadcasts “loser.” At the end of the day, is there anything I can write short of “this person is a genius” that won’t be wildly misread as a secret code for dork? My goal is to write well considered, high information letters for non-geniuses who clearly deserve jobs and admission to graduate school. So, please, tell me the secret code! I’m tenured and I still don’t know it!

Written by fabiorojas

November 30, 2010 at 12:57 am

Posted in academia, fabio

abolish letters of recommendation?

Fabio

Over at the evil twin blog, Organizations and Markets, there’s a debate about the usefulness of letters of recommendation (here and here). Critics usually say the following:

  • Inflation – you don’t want your student to look bad so every student is “the best in 30 years.”
  • Fear of reprisal – If you say something bad, you may be legally liable, or your student will hate you.
  • Misinterpretation – You try to say something serious, but it gets twisted. Example: “hard working” is often interpreted as “not imaginative.” How hard working became a bad character trait is beyond me, but people do read letters that way.
  • Cynicism and office politics – You say nice things just to get rid of people you don’t like.
  • Favoritism – you like your students for personal reasons, not accomplishment.
  • Low added value- what do they tell you that past performance/test scores doesn’t tell you? Aren’t interviews informative enough, especially those long multi-day interviews in academia?

The critics are right – letters are meaningless for these reasons. In fact, private firms usually don’t even bother and will ask for references who get called, because phone conversations are a bit more candid than letters.

But I will stand by them when it comes to faculty hiring, especially junior faculty. Why? Faculty hiring is about finding people who are exceptional and some people may work in unorthodox ways. A person may do work that is very interesting and important, but doesn’t come with the standard seal of approval (the “right” journal or the “right” adviser). Unless you relegate promotion and hiring to journal and book editors, you probably value letters for exactly this reason. In that case, a letter may help explain the importance of work to non-specialists. I still believe that most letters are not helpful. However, for that special 1% that you’re searching for, a rare non-puffed & thoughtful letter can be exactly what you need.

Bonus round: Read Deirdre McCloskey’s take on letters of recommendation. She’s totally against them: The Insanity of Letters of Recommendation. Also, an essay on intentional misreadings of letters. Hat tip to Dick Langlois.

Written by fabiorojas

November 29, 2007 at 1:39 am

Posted in academia, fabio

feeding the bureaucratic machine: letters of rec and other documents

At orgtheory, we’ve repeatedly discussed whether letters of recommendation (LOR) are useful or not.  For some committees and tasks, LOR can help decision-making a lot – for example, undergraduate admissions, graduate school admissions, and tenure for professors.  But for other committees and matters, LOR serve a per forma function and are less likely to impart information useful for decision-making.  Worse, LOR can increase workloads for letter-writers, especially when they are at institutions that do not offer support for this kind of bureaucratic task.*

Nonetheless, committed letter-writers dutifully carry out their responsibilities, especially since LOR can be consequential.**  As Fabio, olderwoman, David Meyer, and others have pointed out, writing effective LOR is an art, with carefully coded language that require fluency on the parts of both letter-writers and letter-readers.  Otherwise, what one letter-writer might view as effusive could be interpreted as a lukewarm endorsement by readers; hyperbolic letters with hierarchical rankings ( i.e., “best student ever”) might be considered suspect.  But what if LOR and related documents were brutally honest about their contexts, rather than merely following convention?

A recently published novel Dear Committee Members takes aim at these bureaucratic documents and their institutions’ crumbling support.  Julie Schumacher‘s novel unfolds as a series of documents composed by a professor for various audiences.  Check out the following reviews for tidbits.

Read the rest of this entry »

Written by katherinechen

November 11, 2014 at 1:09 pm

Posted in academia, books

Tagged with

letter of recommendation theory

There’s a pretty wide spread feeling that letters of recommendation are very inefficient. Scatter has a very good discussion here, check out olderwoman’s comment. Leiter Report, a professional philosophy blog, has a very nice discussion of how people may not know the “code” when writing the letters, and thus unintentionally damage their students. On this blog, I’ve proposed eliminating letters of recommendation in many cases.

So I got to thinking, to have any decent opinion on letters of recommendation, you need at least a theory of how they operate. Here is what you need to know:

  1. “Codes”: does the letter writer know the formal and informal standards for writing letters?
  2. Incentives I: Does the letter writer have any incentive to write a truthful letter? Is there any reason to belive that letter inflation can be contained?
  3. Incentives II: Does the letter writer have any incentive to write a proper letter if they know the “codes?” Even if they are honest, why would people take the time to write letters?
  4. Validity: Do letters actually predict performance?
  5. Value added: What does the letter add that is not obvious from observed behavior?
  6. Costs: What is the abosolute and relative cost of letters in terms of processing and managing them?

Here’s what I think are the answers:

  1. Obviously, there seems to be a huge variance in letter writing codes. In my own case, I always thought that “hard working” was a good and rare attribute. Omar points out that the wider academe sees this as a signal of low quality (i.e., they compensate for lack of talent with extra effort, which is somehow a bad thing). It’s clear that simply getting the form and filling it out is insufficient.
  2. The folk wisdom is that most fields suffer from letter inflation. I am not sure what incentives people have to write accurate letters. People say “I don’t trust letters from so and so.” But who cares? Will that change so and so’s behavior? Not a chance. It probably punishes the applicant.
  3. There is little evidence that people will take the time to write lengthy and carefully crafted letters. Most seem generic. I’ve seen many files that are incomplete. The faculty simply fail to write the letters.
  4. There’s concrete evidence here. Letters are a poor predictor of performance in the settings where they have been studied. Here’s a clip from an article in the journal Public Personnel Management : “Even though references are commonly used to screen and select employees, they have not been successful in predicting future employee success (Muchinsky, 1979). In fact, the average validity coefficient for references is only .13 (Browning, 1968; Mosel & Goheen, 1959). This low validity is due mostly to four main problems found with references and letters of recommendation: Leniency, knowledge of the applicant, low reliability, and extraneous factors involved in the writing and reading of letters of recommendation.” There’s a number of studies in psychology and medicine about letters and future perfomance. The studies show letters aren’t very good. They’re glorified junk.
  5. In most cases, the value added is small. For example, don’t you know enought about a student from the 30+ college courses reported in the transcript? And their standardized tests, often taken multiple times? And the writing sample? That’s a huge amount of information and better than what almost all letters contain.
  6. Costs: It takes time to write letters, ship them, recieve them, sort them, and read them. Lot’s of time. There’s storage and postage. There’s also the time it take students to hound faculty to get the letter done and even then, too many job and grad school applicants don’t have complete files that are often discarded because they could only get 2 of 3 letters. There’s a significant cost.

That’s it in a nutshell: People don’t know what letters are about, they have incentives to write lousy letters, letters don’t predict performance, and there’s a non-trivial cost. Why on earth do we keep doing it?

If you want my “subtle” opinion, mandatory letters should be abolished in most educational settings. Students should be encouraged to submit letters only if they feel that there’s something that’s not clear from the transcript (e.g., a semester of bad grades due to death in the family). Letters might also be required in cases where the committee reading papers truly needs external evaluation. For example, junior job canidadates often have thin publication records and non-specialists might find it hard to understand what’s good in a dissertation. But in most cases, we have a pretty good sense of what’s happening with a student.

Written by fabiorojas

February 11, 2010 at 4:34 am

Posted in academia, fabio

time to divorce your adviser?

We recently discussed toxic advisers, who do more to harm than help. Most of the time, it is best to just tough it out. But some times, it becomes really, really toxic. I have one friend, a brilliant man who knew about eight languages, who simply could not deal with the adviser. It got so bad he fired his adviser. Literally, one semester later the dissertation was defended and he soon got his PhD.

I never did figure out why the relationship wasn’t working, or why my friend’s adviser was so toxic. But I did learn a simple lesson – some advisers are preventing students from finishing and you need to get a divorce.

Should you get an adviser divorce? As with any other relationships, it depends on a few factors:

  1. Make sure the relationship is truly toxic. Be tough and take criticism. That is normal. But it is not normal to yell at students, it is not normal to ignore a student for a year or more, it is not normal to refuse to write letters of recommendation, it is not normal to tear down a student so bad that they can’t constructively improve their work.
  2. Make sure that there are enough other non-toxic advisers in your program and that at least one knows about the situation and is willing to take you on. And don’t be shy. Your career and degree are slowly draining away.
  3. Make sure that the potentially new adviser won’t make you redo everything from scratch. Be blunt and ask up front.

It’s a rare thing. Most of the time, switching advisers is more trouble than it is worth. But if you’re like my friend and semester after semester nothing happens when you hand in drafts, it may be worth thinking about.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street  

Written by fabiorojas

May 11, 2017 at 12:30 am

i am so sorry, the GRE is still a valid tool in graduate admissions

A recent Atlantic article by Victoria Clayton makes the case that the GRE should be ditched based on some new research. The case for the GRE rests on the following:

  1. The GRE does actually, if modestly, predict early graduate school grades and you need to do well in courses to get the degree.
  2. Many other methods of evaluating graduate school applications  are garbage. For example, nearly all research on letters of recommendation shows that they do not predict performance.

To reiterate: nobody says GRE scores are perfect predictor. I also believe that their predictive ability is lower for some groups. But the point is not perfection. Th point is that the GRE sorta, kinda works and the alternatives do not work

So what is the new evidence? Actually, the evidence is lame in some cases. For example, Clayton cites a 1997 Cornell study claiming that GRE’s don’t correlate with success. True, but if you actually read the research on GRE’s there have been meta-analyses that compile data from multiple studies and find that the GRE does actually predict performance. This study compiles data from over 1,700 samples and shows that, yes, GRE does predict performance. Sorry, it just does, test haters.

Clayton also cites a Nature column by Miller and Stassun that correctly laments the fact that standardized tests sometimes miss good students, especially minorities. As I pointed out above, no one claims the GRE makes perfect predictions. Only that the correlation is there and that is better than the alternatives that simply don’t predict performance. But at least Miller and Stassun offer a new alternative – in depth interviews. Miller and Stassun cite a study of 67 graduate students at Fisk and Vanderbilt selected via this method and note that their projected (not actual) completion rate is 80% – much better than the typical 50% of most grad programs.

Two comments: 1. I am intrigued. If the results can be replicated in other places, I would be thrilled. But so far, we have one (promising) study of a single program. Let’s see more. 2. I am still not about to ditch GRE’s because I am not persuaded that academia is ready to implement a very intensive in-depth interview admissions system as its primary selection mechanism. The Miller and Stassun column refers to a study of physics graduate students – small numbers. What is realistic for grad programs with many applicants is that you need to screen people for interviews and that screen will include, you guessed it, standardized tests.

Bottom line: The GRE is far from perfect but it is usable. There is no evidence to systematically undermine that claim. Some alternatives don’t work and the new proposed method, in depth interviews, will probably need to be coupled with GREs.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

March 16, 2016 at 12:01 am

t-t assistant professor opening at CCNY

My dept. is hiring for a t-t assistant prof line – please download this: (Job_Announcement_2014 final) or see below.  Best wishes to everyone on the job market.

Read the rest of this entry »

Written by katherinechen

October 1, 2014 at 12:41 pm

Posted in academia, sociology

Tagged with

how to manage letter-writers who lack administrative support

Nearly everyone agrees that letters of recommendation are a lousy system that provides little information, rewards high-status connections, and provides lots of opportunities for recommenders to inadvertently damn their recommendees (see past orgtheory discussions here, here, and here). Yet for now, at least, we’re stuck with them.

Job season is approaching, and grad students are getting ready to request letters. If you’re at a well-financed program, your department will have an administrative person responsible for making sure your letters reach their many destinations. If you’re at a program like, well, mine, your letter-writers will find themselves sending 40-60 (or more) letters in a variety of formats for each of several students on the market. Needless to say, this is a big administrative pain.

A graduate student about to go on the market (okay, it was the awesome Josh McCabe, hire him!) asked this week about the best way to manage all these letter requests. Here’s my thoughts:

1) If you have the money, using Interfolio would be simplest, safest, and easiest for your letter-writers. They can upload letter(s) once and you can send them wherever, and whenever, you want. This is what I did when I applied for a handful of fellowships last year (the political economy of letters doesn’t end once you have a job). But Interfolio costs $6 a pop, and I’m not comfortable asking grad students to pay that for each of those 40-60 applications, possibly over multiple years.

2) Otherwise, there are a couple of principles to remember, beyond the general stuff that you’d do even if your department has administrative staff to handle letters.

– Ask what you can do to facilitate the process. Different people like different things. Personally, I don’t like complicated job spreadsheets, which can be idiosyncratic and hard to read. What I like is a list of basic info in the email — contact person, email/website/snail mail address, deadline (in bold!), link to the ad, a phrase or two on the job (e.g. “organizations, quant preferred”), and whether you’d like the research or teaching version of your letter — sorted either by deadline or by type of submission (website, email, or hard copy). Others may differ.

– Batch, batch, batch. There is nothing worse than receiving those 50 requests one at a time. Aim to send your requests once a month, maybe once every two weeks during the busiest season. Yes, there will be jobs posted that may make this impossible sometimes, but to the extent possible, group your requests.

– Manage up. FIgure out how your letter-writers work, and what you need to do to stay on top of them. If they are totally organized, you may not need to follow up. Personally, I just reply with “Done” when I’ve sent a batch out so the student knows when it’s been taken care of. If you don’t completely trust your recommenders to be on top of deadlines, you may want to mention in your request that you’ll check in a week before the next deadline to confirm. In some cases, online application systems will tell you whether letters have been submitted, which will allow you to avoid excess emails. But this is not always possible, and it’s totally reasonable to ask for confirmation that letters have actually been sent.

– Stay organized yourself. There are a lot of bits and pieces to manage on the job market. Keep track of what you’ve done and what you need to do, so you aren’t inadvertently making multiple requests that a letter be sent to the same place.

Sending letters, while a pain, is part of the job of faculty and almost everyone recognizes that. Perhaps we’ll eventually get a centralized system that will eliminate this problem, or, even better, abandon letters. But until then, there are better and worse ways to manage the process. (Also, are we the only department out there where faculty send all the letters themselves, or is this fairly common?)

Written by epopp

July 12, 2014 at 4:50 pm

Posted in academia

minorities and academia: some further thoughts

When thinking about increasing the presence of under represented minorities in the professoriate, I think of the pipeline process model. Roughly speaking, a pipeline process suggests that something happens in multiple stages. The immediate consequence of the model is that if you want X to happen you have to make sure that all the stages that make X are working properly. In terms of faculty diversity, that means recruitment to graduate school, professional training, job placement, career development, and the tenure process.

A while ago I reviewed evidence from ASA reports showing that the pipeline is leaky. On the one hand, graduate programs seem to recruit a fair number of minority students. Then, once training is complete people seem to do well getting the jobs. Then, there is a massive drop in the pipeline as people go up for promotion.

Now that I’ve been on the job for a while, I think the following is happening: the core faculty of the PhD programs are not working with minority PhD students. They are admitting students, awarding degrees, and writing letters of recommendation, but they are not collaborating with students in ways that lead to publications and grants. In other words, most successful students work with faculty who “get them started” while their own research takes a little time to develop. My hypothesis is that if you looked at PhD minority students they are way less likely to co-author with faculty and that they are less likely to receive an offer of co-authorship. I’d also hypothesize that this gap is largest for top tier journal publications. This will be small or non-existent in areas focused on race and ethnicity. In other words, when faculty build teams to shoot for that ASR or AJS publication, the minority students come last for invitations, except in race & ethnicity areas. I didn’t think this is conscious, but this might be happening and explains the drastic leaking throughout the later stages of the pipeline.

Am I right? If you are a faculty member at a top 20 or 30 program in your field, the test is simple. Look at your list of co-authors for your big papers. Look at your list of minority students. Look at the overlap. Use the comments section.

50+ chapters of grad skool advice goodness: From Black Power/Grad Skool Rulz

Written by fabiorojas

April 30, 2014 at 12:01 am

Posted in academia, education, fabio

post doc position on social media and activism

I’m really happy to announce a new post doctoral position here at Northwestern University on social media and activism. If you’re interested, please apply early. The application deadline is March 2nd! Read the rest of this entry »

Written by brayden king

February 4, 2014 at 9:59 pm

the tragedy of the fabios

For most of my years as an academic, I’ve been lucky in terms of time management. While I felt stress, it was not be because I was out of time, except for those times when I had a baby at home. Then, this semester it really, really hit – professor overload. Every month, I’d get four to five reviews for journals. Just in the last semester, I’d get about one book manuscript review a month. Then, book review requests, letters of recommendations, and oodles of other stuff. I haven’t even gotten to my regular committee work, teaching, research, or mentoring graduate students. Of course, this is a blessing. It means that people value my input on various issues. But it’s also imposing a cost.

I mentioned this to a colleague, who said it was a tragedy of the commons issue. Professors are a free resource. The university pays our salary, but we work for others for free. Journal reviews are free (in soc, at least). Book reviews (for presses and journals) are free or very low cost. We don’t charge for our time on committees. Students who ask for a paper regrade do so for free. The solution is obvious – charge by the hour! So the next time someone asks you to do committee work, be ready with your rate and the requests will be more considerate.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

January 13, 2013 at 4:08 am

Posted in academia, fabio

hiring and retaining faculty

Besides prepping for teaching, many orgheads are gearing up for the academic labor market, either as job applicants or as members of hiring committees.  On both sides, it’s blood, sweat, and tears, particularly if committees are committed to hiring colleagues they wish to tenure.  Job-seekers examine ads, network, and pull together packets, including letters of recommendations.   Hiring committees go through hundreds of applications, arrange on-campus visits, and navigate bureaucracy to get the necessary paperwork in order.  Both sides want a “good” fit and try to ask questions accordingly.

What is often overlooked – particularly on the side of the workplace hiring practices – is that individuals are never “just” individuals.  People are embedded in a network of relations that they seek to sustain or expand.  In some cases, this network involves a dyadic relationship, i.e. a job applicant may be part of a couple, or a larger unit such as a multi-generational family.  Or, the job applicant may be seeking an environment where s/he can form and sustain a network for a vibrant social life.

Back in 2008, Stanford University’s Clayman Institute for Gender Research issued a report about university hiring practices, with a focus on dual-career couples, that should be of interest to those on the market and those seeking to hire and retain colleagues.  Besides reporting the results of a survey of academics’ experiences in selected universities, the report outlines some steps that universities can take, such as participating in agreements to consider a dual hire among departments within the hiring university and with near-by universities or offering shared tenure-track/tenured positions.  In addition, the report notes the needs of single academics, who may be negotiating caregiving for children and aging parents.  The report also addresses the concerns of same-sex couples given different states’ policies on domestic partnerships and same-sex marriage.

What I found particularly fascinating was how the report illustrated how hiring committees can anticipate the likelihood of someone being paired with another academic, based on academics’ propensity for “disciplinary endogamy” (partnering within the same academic discipline).  With this knowledge, the hiring committees can try to investigate options that increase the likelihood that the desired applicant will accept an offer and remain with the university:
On p. 32, box 5: “A well-known physics department has advertised a job. The university has the resources to hire a partner, if sufficiently qualified. Because this is a junior-level position, time is of the essence, and the department chair would like to know whether candidates who make it onto the short list have partners who may need to be considered for a job. Even without asking, the chair can have a sense of how likely it is that a particular candidate will have a partner. One candidate on the short list is a woman: Because she is a woman, there is a 40 percent chance that she has an academic partner (Figure 2). Because she is a scientist,
there is a 48 percent chance that she has an academic partner (Figure 12), and if she has an academic partner, there is an 83 percent chance she is partnered with another scientist
(Figure 13). Because the candidate is a physicist, there is 58 percent chance that partner is also a physicist (Figure 14).”

Thoughts on what universities can do to better hire and retain faculty?  Put them in the comments.

Written by katherinechen

August 10, 2012 at 5:17 pm

Posted in academia

what is it like to be a professor?

Got a recent email from Ben. He bought the Grad Skool Rulz and is seriously thinking about graduate school. He is curious about what it’s like to be a working academic. Smart guy. Ask now.

Here’s how I’d describe it. There are three stages to being a tenure track professor professor: trainee (grad  student), probate (tenure track), and zombie (tenured prof). It’s important to recognize that this does not describe the majority of academics. These days, the average college instructor is an adjunct (part time) instructor). Some people like this arrangement, especially clinical faculty, such as lawyers, who are hired to teach the occasional course in a professional school. For most, however, the adjunct career track is low paid work that requires “freeway flying” between far flung campuses.

But let’s stick to the tenure track because that’s what Ben is shooting for:

  • Trainee: Graduate school is uneven. The first two years are courses, then you have an extended period of self-directed study and research. You also have to learn to be an adult. Learn to do your work without a boss or deadline. Don’t get published and you’ll get a career failure. Don’t do your dissertation and you have nothing to show for your work.
  • Probate: Assistant profs are paid and have a high stress level. It usually takes a long time to execute a project and get it published. You may have five or six years, but it goes by quickly as you work on these big projects. The difference between trainee and probate is quality. The dissertation is a student exercise, so a competent work will get approved. In contrast, the competition is tough for journals and publishers. Top publishers routinely reject 90%+ of submissions. The other big difference is teaching. Research faulty teach 4 courses a year, though they can buy some out. Liberal arts faculty do more.
  • Zombie: You have enough experience with publishing and you’ve managed to balance teaching and research demands. The killer here is committee work. If you are in a research department, you also have graduate student training.

My days are usually divided into teaching days and research days. Personally, I try to cram all classroom time, office hours, and grading into a few long days. It’s about minimizing transaction costs. So a few days a week, I roll into campus late in the morning and teach these long seminars and meet with students. Since I pack a lot into a few days, this may go into the evening, especially if I have afternoon/night classes.

Research is a bit different. Since I am a multi-method researcher, doing research can mean very different things. Currently, I am involved in some surveys. So I spend time writing grants. At other times, I do ethnography. So it’s about making travel arrangements or deciphering field notes. Then, at other times, I may be programming. But unless I am traveling, research usually means reading current research,,working with data, and writing papers/book manuscripts. That entails sitting in front of computers for a long time.

Teaching and research are interrupted by committee meetings. This is utterly boring. Sometimes the meetings are boring and crucial (like hiring or promotion), or boring and not crucial (listening to an administrator tell us about the latest mission statement). Regardless, they are a necessary evil of the academic profession. I am lucky to be in a program where meetings are kept to a minimum.

At the zombie level, you also do a lot of evaluation. Senior professors are asked to review papers for publication, book manuscripts and write letters of recommendation, including tenure letters.

Finally, Ben asked about research topics. This is easy. Just pick up the journals, books, and recent dissertations in your area. Read them and see if they inspire you. If they leave you cold, then academia probably isn’t for you.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

May 18, 2012 at 12:01 am

sociology and management phd program admissions – comments and open thread

My Facebook news feed is filled with colleagues commenting on graduate program admissions. Let me take this opportunity to make a few comments and open it up for discussion.

1. If you are applying to IU, I am quite sorry. We probably won’t be offering you admissions. It’s just a fact. We reject the overwhelming majority of applicants, including many who will go on and have great careers in sociology.

2. The rest of my comments are directed at faculty who are serving on admissions committees. First, departments vary in their strategies and procedures. Here are IU, we read through every single application. Of course, some folders get more attention than others. The folks with rock bottom GRE’s and a 2.9 GPA won’t get more than a quick glance, if that.

We do read a lot of folders in detail because IU employs a sort of “Moneyball” strategy. We look for diamonds in the rough. A lot of our star students, who go on to dominate the job market, were high performers at relatively low status schools. So we don’t rely on a steady feed of polished applicants from the West Coast and Ivy League. We find the gems from the liberal arts and public schools of the Midwest and South. It’s a strategy that requires reading folders closely, but it pays off in spades. IU has great students.

3. It is hard to develop meaningful distinctions with a certain class of students. You might call them the stereotype sociology undergrads. The profile is that they major in sociology and have a decent GPA. They also have decent verbal scores, but bad to miserable math scores. Unless they can make the case that their math score is an aberration (e.g., they did well in freshman calculus), it’s hard for them to move to the top of the pile.

4. Admissions committees often have trouble interpreting applications from foreign countries. Sometimes it’s language, sometimes it’s simply a different grading system. Also, you have to work extra hard to distinguish between students who are more interested in migration than the academic career. Some regions have a reputation for less than trustworthy credentials. That’s why it’s good to consult with colleagues from those nations  if there’s a candidate who might be a good fit. If I were to advise foreign applicants, I’d insist that they show that they “get it” (i.e., understand academia) and have a credible signal of commitment to academia.

5. Letters of recommendation: My opinion of letters continues to slide over time. The more I do admissions, they more flaws I see. First, there is little variance among letter writers. Second, there is a double selection effect. Students only approach profs who like them and who have given them good grades. Third, a lot of letters are obviously lame. For example, I have read many letters that insisted the student was top notch (top 1% or 5%), yet the GRE’s and GPA were clearly atrocious. Fourth, a lot of students, especially those in low status schools or who have non-academic employment, have letter writers who are not in a position to write honest and thoughtful evaluations. I’ve seen a lot of letters by bosses, academic advisers, project supervisors, and so forth. I don’t count it against students, but it doesn’t help.

6. The 1% of letters:  Still, every once in a while, there’s a letter that makes a clear and compelling case for a student. The profile of the letter writer is that they are an active teacher and researcher. They provide some concrete evidence that the student is actually exceptional, or that they are better than the transcript. They often have extensive experience with the applicant, or they can explain why the performance in one course is remarkable. Sadly, I’ve read only four or five letters that fit this mold, out of hundreds. The rest are generic and uninformative.

7. Random thoughts: Statements are good for sorting students, but only broadly. If the applicant is interested in social work or activism, academic sociology isn’t a good fit. I read transcripts carefully to spot praiseworthy or suspicious behavior (e.g., lots of withdrawals, hard courses, upward trajectories). Writing samples are good measures on general writing, but still, I am reluctant to make a decision on a product that was often not originally intended to be research sociology (e.g., a term paper in history or a policy report).

Add your admissions questions and remarks in the comments.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

January 30, 2012 at 12:09 am

Posted in academia, education, fabio

how i pick grad students

Don’t take this post personally – I haven’t been on a graduate admissions committee in a while. But I wanted to start a thread on how sociology programs pick doctoral students.

Things that impress me:

  1. Overall excellence: Like nearly anyone, I am impressed with applicants who have a high GRE score and GPA.
  2. Courses that are tough: I am impressed when people do well in courses that are regarded as demanding. In the sociology major, these would include social theory and statistics. Outside the sociology major, I am impressed by foreign languages, philosophy, and anything that requires non-trivial amounts of mathematics. I also give slack for the challenge. The B+ in advanced Arabic is probably more persuasive to me than an A in intro soc. For this reason, I read transcripts very carefully.

Things that sometimes impress me:

  1. Research papers/writing samples: If they are well written and address non-trivial topics, then I impressed.
  2. Choice of major: If you can maintain a decent, though not perfect, GPA in a really challenging major, I am impressed. I am not impressed by bad performance in a tough major. I also expect near perfection from applicants in notoriously easy majors.

Things that usually don’t impress me:

  1. Statement of purpose: I usually find this useful only as a test of basic language skills. It also communicates that the applicant is a good fit with respect to research interests. But these considerations rarely come into play because I can usually tell what I need to know from your transcript and test scores.
  2. Letters of recommendation: As I have argued many times on this blog, most research, with a few exceptions, shows that letters of reference are poor predictors of future performance. I quickly scan letters for red flags to weed applicants who are clearly not suited for graduate school. But aside from that, I pay very little attention to them.
  3. The college you came from, unless it is known as an incredibly demanding/undemanding place.

Stuff that I ignore:

  1. CV’s/resumes
  2. Personal stories/life experience
  3. Work experience, unless it is obviously related to your research and it’s really unique.
  4. Internship/Summer research experience. Nothing wrong with those activities, but I don’t have any personal experience or research showing me that these predict success.

I know that people disagree with me on many points. I make no claim that other faculty use these criteria. I’d be interested in your opinions with regard to evluation of graduate school applicants.

Written by fabiorojas

May 6, 2011 at 12:15 am

Posted in academia, fabio

grad skool rulz #25 – the job market

Previous grad skool rulz.

Get the entire book – Grad Skool Rulz: Everything You Need to Know about Academia from Admissions to Tenure – for only $2. You can read it on personal computers, Nooks, Kindles, iPads, and smart phones.

This post was originally written for graduate students in sociology. The advice applies to other fields, with suitable modifications. E.g., many fields have short interviews at annual conventions during the winter break. Sociology’s job market happens in the fall, so there are no Christmas time interviews.

Initial Remarks: The job search process is harrowing for academics. Unlike other professionals, such as doctors or lawyers, there is little guarantee that a person completing their terminal degree will land a job teaching and doing research in their area. At a top medical school, the question is if you will get the residency of your choice. At a top graduate program, it’s often doubtful that someone will be offered a job at all. Despite this difficult situation, I believe that you can prepare yourself and greatly improve the chance that you will get an academic job. What follows are my opinions on junior level academic job searches, with an emphasis on sociology.

Question 1: Should I go on the job market?

Answer: You get 1 point for each “yes” to the following questions. The more points you score, the better prepared you will be for the job market. As usual, adjust for your field. For example, in short clock fields, where you leave after 4 years, you probably won’t have the chance to publish much.

  • Have I finished my dissertation proposal?
  • Have I completed the data collection for my dissertation?
  • Have I completed at least one polished chapter of my dissertation?
  • Do I have more than one chapter of my dissertation completed?
  • If I get a job, can I complete the dissertation by the summer before I have to start?
  • Do I have a published article in a reputable refereed journal?
  • Do I have multiple articles?
  • Are any of those articles in the top journals?
  • Do I have a book contract? (this often counts for two points)
  • Do I have the support of my committee? (counts for multiple points)
  • Do I have teaching experience? (counts for more if you want a liberal arts position)

Of course, you should always consult with your committee so that everybody is aware of your progress and you are get feedback on your writing. If you have published an article, make sure your committee knows about it. If you have decent drafts of some dissertation chapters, make sure your committee sees them.

I also note that few people can answer “yes” to the all of the questions. But you need to have *something* going for you.

Question 2: When should I think about the job market?

Answer: In sociology, the job market starts in September. So start thinking and planning the spring or summer before the market. As you will see, there is a bit of paper work, so it behooves you to plan this ahead of time. Many fields have job markets that take off during the winter, so you have to start planning everything in the fall.

Question 3: How does the job market work? Once again, written for sociology. Adjust for your own field.

Answer: It goes something like this…

  1. In the spring, summer and fall, department chairs and deans will make decisions about hiring. If they decide they need people, they will advertise in the ASA job bulletin and other forums. Some departments will “scout” at the ASA meetings.
  2. Applications are due in the fall. Many are now due in late August, September and October.
  3. Your application has to have a cover letter, a CV, writing samples and 3 letters of recommendation. Work on these during the summer, so it’s ready to go in the fall. There are books that give great advice on cover letters and the rest of the paperwork. Ask your committee for help as well.
  4. The search committee first weeds applications based on very broad criteria by about 50%. This is based on school reputation, research areas and other easy to observe factors. Then applications are weeded by what the department really needs or wants. This produces a “long short list” of 15-20 names. The “short list” is created after close scrutiny and reading. Then people argue over who to invite for a face to face interview. This happens in Fall and early winter
  5. The candidates are flown out to the campus. Usually, 2-3 candidates per position. You have to give a research talk and meet people so they can see what you are like. It’s very personal at this stage. The visit includes a “job talk” – which is a 30-45 minute presentation of your research in a public forum.
  6. After everybody interviews, the department makes a final choice. Sometimes they don’t get the first choice and will go after second/third choices. This process can take many months. Some departments will choose not to hire people.

Question 4: How do I write a cover letter?

Answer: It’s pretty standard. Your letter is addressed to the search committee or department chair, as listed in the advertisement. The first paragraph explains who you are and what job your are applying for. “My name is Fabio Rojas. I am a doctoral candidate at the University of Chicago and I am applying for an assistant professor position at Indiana University as listed in the ASA bulletin.”

For research intensive schools, the next two or three paragraphs explain your research and publications in terms that people in your discipline can understand. A little self promotion is ok. Mention publications in fancy journals, or that you won dissertation of the year award. Don’t mention people who endorse you (“My adviser is highly regarded XX.”) The next to last paragraph should mention teaching, For teaching schools, it’s the reverse. The big chunk of the letter is about teaching experience. Research accomplishments are second. Overall, cover letters are about 2 pages, sometimes 3.

Question 5: How do I prepare my self for an interview?

  • Be prepared. If you are prepared, then you will be relaxed and you will give a better impression.
  • People will ask you predictable questions. “What will you teach?” “What will you do after your dissertation?” Prepare some answers. These are obvious questions.
  • Learn about the dept. you will visit. Read the web site, look at some papers published by faculty. I’m always fascinated by what other people are working on.
  • Be nice. Even if you have an intellectual disagreement with someone’s research, be open and generous when you meet them.
  • Humor and demeanor: Be “vanilla” – don’t swear or be sarcastic. Your friends may find you funny, but somebody who doesn’t know you might find your jokes raw and wicked humor off-putting.
  • Never badmouth anybody. If someone asks you a question about a professor you hate, like: “I heard Professor X is awful.” Simply say, “Professor X has always been kind and generous towards me.” Or if you can’t say that without hysterically laughing, say, “Professor X’s research is really admirable.”
  • Be honest. It’s better to emphasize your good points rather than mislead. If you hate math, don’t say you can teach regression. If you think post-modernism is for the birds, don’t say you’ll teach cultural sociology. Just move the conversation towards your strong points.
  • Appearance: You don’t need an Armani suit to succeed but wear nice clothes. Have them dry cleaned. Make sure they fit. Guys should wear jacket and tie. Ladies should wear blouses. In our modern age, the ladies can wear slacks. Have your hair and nails cut, brush your teeth, etc. Simple things go a long way. Trust me.
  • Other etiquette. Use common sense – be nice toward people, don’t get drunk during social events, take a real interest in others.

Question 6: See the next grad skool rulz (#26).

Question 7: Bad situations. Sometimes interviews have awkward moments. For example, in the real world, some people will make sexual advances towards others or engage in some form of harassment. If the behavior is mild, it’s probably best to ignore them. Life has bad moments you have to endure. If it’s more serious, then you should definitely say something like, “I don’t think that’s appropriate.” If the behavior is really off the wall, feel free to contact the department chair or to consult with someone you trust. It’s often the case that boorish behavior is part of a larger pattern, and others will know how to handle it, or at least make things tolerable until the end of the interview. The key is to get help and not let things get out of control.

Question 8: Success. After the interview, the department will decide who will get a job offer. This is out of your control – once you’ve completed the interview, it depends on budgets, personalities and other factors. In some departments, the chair makes final decisions and in others, committees make the decision. In most cases, the offer has to be approved by the dean or some other academic manager. The department chair usually does the work of contacting job candidates and formally offering the job. An offer consists of:

  • A position (assistant professor, associate prof, etc)
  • Salary
  • Equipment (computer, transcriptions devices, etc)
  • Eesearch assistance (money or assistants)
  • Other goodies (summer support, course releases, research funds, etc.)

Get this in writing! You can negotiate a lot of stuff and ask for more, but you might not always get it. Ask your committee and other job seekers what the market will bear. Get everything in writing. You usually have a few weeks to a month for negotiations. When you are done negotiating, sign the contract and mail it back. Now finish your dissertation!

Question 9: Failure. Sometimes you fail to get a job – and this is a real possibility in the academic market place. Unlike the other professions, there are relatively few academic employers and excellence in research does not always translate into success. There are two possibilities you must consider:

  • You have done everything right but suffered bad luck. This is quite common. The average academic job seeker only has two or three interviews and gets a single offer. This single opportunity could be thwarted by events beyond your control. An unexpected budget cut could mean your job was eliminated at the last minute. Maybe there is an unexpected conflict over the hire. There are a million other reasons you don’t get an offer – and you will never know why!
  • You are screwing up. This is also a real possibility. Ask yourself how you might have given a bad impression or otherwise made a mistake. Here are some common errors: (a) poorly prepared/delivered job talk; (b) you are no where near completing your dissertation and everybody knows it; (c) you are rude towards people when you visit and they are insulted – this is quite common; (d) your research is hard to sell; (e) you are “packaged” incorrectly – for ex, your committee thinks you are God’s gift to quantitative research but you barely understand regression; (f) you flubbed basic questions such as what you will teach and what your future research will be like. Fortunately, most job search mistakes can be fixed and you will improve your odds the next time around.

Overall, the academic job search is a dragged out, often arbitrary process. The bright side is that you can still prepare and fix your mistakes if things don’t work out. You do have a great deal of control over what happens to you.

Written by fabiorojas

July 12, 2010 at 4:48 am

staying away from the grindstone

I’ve been asked to write a bunch of letters of recommendation this semester.  Engaging in this exercise brought back to mind an amusing debate that I had with Fabio in what seems like a thousand years ago. In this exchange Fabio defended—what I still think is the hopelessly naive—proposition that using terms such as “hardworking” in a recommendation letter was a good thing.  I had the unenviable task of reminding my recalcitrant friend of the fact that certain adjectives are not “positive” on their own, especially if they (automatically as cognitive scientists are increasingly finding) bring to mind their opposite (e.g. effortless talent) and especially if said opposite is precisely the ideal that is (implicitly, wordlessly) favored in the elite nooks and crannies of certain culture production fields (such as academia).

The funny thing is that it never occurred to me until now to consider the possibility that there might actually exist a social science literature on this.  It turns out that (an apparently tiny) literature does exist and that (shocking for some I suppose; absolutely unsurprisingly to me), the phenomenon even has a name (and if you read Merton, you know that if a phenomenon has been baptized then it must be important!).  It is called “grindstone words.”  In the—content and discourse-analytic—research that I had a chance to carelessly glance at (here and here), grindstone words (including “hardworking”) are negatively correlated (in a biochemistry and chemistry sample of letters) with the use of “standout” keywords (which as I noted originally, generally refer to intrinsic or effortless ability).  One of the papers (using a medical school sample) finds that grindstone words are used more often in letters describing women candidates in comparison to male candidates (which is what you would expect if the automatic association of grindstone words was negative and not positive).

So for the love of everything that is holy, and pace Fabio and Steve, don’t pepper (I don’t and I have been able to fill up up three or more single-spaced pages) your recommendation letter with allusions to hard work (although you should listen to Fabio on everything else that he’s ever, ever said; ever.  Oh, except for that Obama’s toast thing).

Written by Omar

December 4, 2009 at 11:47 pm

how do you justify the time?

So I’m about a third of the way through my guest-blogging and have only done three of these. My obvious excuse for not doing more is lack of time. But one thing that has become really salient to me is that the challenge in blogging–at least for me– is less the lack of time than the fact that blogging makes my time-use public. That is, anyone who cares can see that I’m spending some of my time blogging when I could be doing other things. Of course, hardly anyone cares how I spend my time.  But a few people do.  In particular, about a dozen people at any one time are waiting on me for various things (comments on papers, letters of recommendation, referee reports, contributions to coauthored articles, contributions to committees, etc.), and now this is making it painfully obvious that I’m *choosing* not to work on those things, rather than being constrained from doing so.  By contrast, nothing else I do is so public.  I’m curious how you orgtheory guys deal with this problem, or perhaps it’s less of a problem for you because the various demanders of your time understand that you are committed to blogging, so it’s part of the bargain in working with you?

Written by EWZS

October 12, 2008 at 6:06 pm

Posted in uncategorized

grad skool rulz #18 – what professors can do to help

Fabio

Get the entire book – Grad Skool Rulz: Everything You Need to Know about Academia from Admissions to Tenure – for only $2. You can read it on personal computers, Nooks, Kindles, iPads, and smart phones.

I’ve only been on the other side of the PhD for five years and I haven’t had chaired any dissertations, but I do feel that I should at least mention what faculty can do to help grad students finish in a reasonable time, barring a Skocpol style incentive system.

  • Grad students have lives and they need you: It’s easy for professors to get wrapped up in their own publications and promotions and forget that grad students need your time if they are ever to proceed with their own lives. I am not saying you should martyr yourself and spend all time on graduate students, but you should periodically ask “what can I do *this* semester to help my students move along in their career?”
  • Expectations: Like all ventures, explain to your graduate students – over and over – what you expect.  From the beginning. Write it down. Also try to gauge their expectations. If they want an R1 career, make it clear what they will need an exceptional dissertation or a top journal hit. Teaching colleges require less spectacular research but a big teaching portfolio. Be clear on what kind of support you can provide, both socially and academically.
  • Timeliness: This is real important – respond to dissertation drafts and letters of recommendation in a timely manner. Don’t you hate it when reviewer C takes a year to read your paper? Well, guess what? Your students feel the same way. Every semester you fail to graduate someone because you couldn’t take the time to read a chapter literally costs a grad student thousands of dollars in lost income.
  • Calm Criticism: It’s entirely legitimate to tell a student that they need to work hard and do better. However, it’s never useful to do so in a way that demoralizes the student. Be stern and demanding, but be nice, constructive and uplifting. On a related note, avoid changing the goalposts or providing ambiguous advice. Consistency is a virtue.
  • Stability: Academia is full of divas. Don’t be that way. You should be the stable coach who taps into the right emotional pool to help students move on with their lives. Don’t turn mentorship into another stage for acting out your bad side.
  • Reasonableness: Set research goals that your students can acheive and where there can be a reasonable time table for the completion of the project in a few years.. Also be prepared to help students work to acheive those goals, instead of letting them figure it out for themselves.
  • Match students with goals: Notch expectations to ability and career goals. The student gunning for R1 needs an advisor who will demand good work, but the person aiming for community college teaching merely needs to produce a satisfactory dissertation. Also, remember that if you have PhD students, you are probably a respected, if not leading, member of your academic community. You are the best. In contrast, your students may not be.  Most will not engage in the research career that you have. Your talent and career may not be theirs. Set goals that both produce quality scholarship and allow them to work toward goals that match their ability and desires.
  • Gentle Triage: This is tough, but needed. You have to really see which graduate students are willing and able to complete the program. Help people make the decision to pursue academia or another career. On the other hand, don’t “write off” students just because they aren’t perfect. Remember, many leading scholars failed a grad school test or acted like morons 30 years ago in that seminar. Give people second, third and fourth chances. Tolerate people who work differently than you do and don’t automatically dismiss them.
  • Selection:  Accept students who you think you can have productive relationships with based on research focus or personality. It’s ok to turn down students if the fit is bad. This is the flip side of grad skool rulz #7.
  • Face to face time: Get ’em in the office. Frequently, at least a few times a semester. Take ’em out to lunch. Anything to keep them on the wagon.
  • Let them shine: It’s often the case the students apprentice on the mentor’s projects. That’s great, but make sure they complete their own work as well so they don’t look like they’re just your research assistant.
  • They are future professors: Above all, these are adults who have begun a career. Treat them with dignity and respect.

R1 faculty, please add your own advice in the comments, especially if you have a solid track record placing PhD students.

Written by fabiorojas

March 30, 2008 at 3:34 am

merry christmas from orgtheory

Fabio

A very silly holiday greeting to orgheads, especially those needing letters of recommendation.

Written by fabiorojas

December 25, 2007 at 1:17 am

Posted in uncategorized

grad skool rulz #8 – the rest of your committee

Get the entire book – Grad Skool Rulz: Everything You Need to Know about Academia from Admissions to Tenure – for only $2. You can read it on personal computers, Nooks, Kindles, iPads, and smart phones.

Fabio

The last installment was about choosing your dissertation adviser. This week’s topic is how to select the rest of your committee. As with the adviser, there is no “perfect” committee, but you should try to choose people that have some positive traits (see grad skool rulz #7 for the list of good traits). You should also follow these rules of thumb:

  1. Compliment: If your adviser is weak on topic X, choose committee members to fill in the gap. For example, if your adviser is kind of slow with the letters of recommendation, choose someone who is very professional and does things efficiently. Aloof prof X can be complimented by emotional & supportive prof Y. If prof X isn’t up to date on statistical technique, get someone who is.
  2. Compatibility: Professors are human beings – they have their own disputes and you don’t want to get caught up in the tussle. Most profs will keep dept politics out of graduate training, but you should still be careful. So choose people who will get along with each other. If you have heard that profs X and Y have it out for each other, do *not* put them on the same committee. If you must, consult with your chair or the graduate chair to make sure it will be ok.
  3. Transaction Costs: Remember, getting people to agree on anything is hard. Thus, you should minimize the number of committee members. Get the dept rules (see grad skool rulz #1) and figure out the minimal number of people you need on a committee. And stick to that minimal number! There is rarely any benefit to having reader #6, and there’s a chance they could mess you up.
  4. No block heads: It can be hard enough working with your dissertation committe adviser – so don’t stack your committee with block heads. Choose people will work with you and your adviser, not against you. These people are often easy to identify – they make all kinds of crazy demands on qualifying exams or oral exams. They make students cry in office hours. They seem more interested in ritualistic torture of grad students than professional development. Do not, under any circumstances, put these people on your committee.

The dissertation committee is a team that has two goals: training/advising students and helping the student get a job. Most of the work is done by the chair, so make sure people can work with that person to help you develop as a professional and get a job.

Written by fabiorojas

April 29, 2007 at 6:14 pm

grad skool rulz #7 – picking the adviser

Get the entire book – Grad Skool Rulz: Everything You Need to Know about Academia from Admissions to Tenure – for only $2. You can read it on personal computers, Nooks, Kindles, iPads, and smart phones.

Fabio

In previous installments, I’ve discussed how to get through the first half of grad school – courses, exams, friends, etc. The next bunch of “rulz” will be about the second half of grad school – dissertations, advisers, articles, jobs, etc.

This week’s topic: how to select your dissertation adviser, the person who will head the committee that must approve your dissertation. This is very important because it is very difficult to change advisers once you have begun your dissertation and you will need their professional support for a *very* long time. So choose wisely.

Advice in a nutshell: No adviser is perfect, but they need to have at least a few strong suits. Also, the dissertation student-adviser relationship is like any other relationship. If your work style/ professional attitudes don’t match, you should consider other options.

Here is a list of desirable adviser traits. As I said, no one is perfect, but you need *something* to work with. In no particular order:

  1. Placement – A track record of placing recent students in schools you would like to work in. For example, if you want to be a liberal arts teacher, don’t work with someone who disdains undergraduate teaching.
  2. Reputation – A reputation within the profession as a competent and accomplished researcher [Note: I didn’t say “super star.” Just respected within his/her field.]
  3. Authorship/ Co-authorship – A track record of publishing with graduate students in reputable books/journals. The adviser encourages students to publish their own work during the PhD program or shortly thereafter.
  4. Funding – A track record of helping students with funding via grants/research projects.
  5. Usefulness – The ability to offer constructive criticism and praise. One without the other is usually a recipe for emotional disaster.
  6. Accessibility – they are actually around campus so you can consult with them.
  7. Craftsmanship – the ability to see that academic research is a craft that can be taught and developed.
  8. Professionalism – the ability to complete administrative tasks such as writing letters of recommendation for jobs and fellowships.
  9. Boundary control – the adviser does not overstep personal boundaries and treats you as a colleague in training.
  10. Expertise – the adviser knows and/or cares about the are in which your are working.
  11. Personality Match – Make sure your adviser can tolerate your persona. For example, if you are very chatty and need feedback, make sure your adviser can deal with this. They don’t need to be chatty, they only need to be able to tolerate chattiness in others.
  12. Intellectual Style Match – Make sure you can handle the “style” of your adviser. For example, if you are going to write a tightly argued statistical dissertation, don’t pick the guy who reads Foucault all day. A loner shouldn’t work with an adviser who does all group projects. However, if you are willing to learn, you can get a great deal from somebody with a different “style” if you can make some compromises.
  13. Social match – Make sure your adviser has a reputation for liking/tolerating people with your social/intellectual characteristics. For example, some folks really feel more comfortable working with people of a certain gender, or they prefer only ethnographers. Don’t be on a crusade to change other people’s personalities. But be open minded – some people only appear rigid on the outside and can be rather open minded when approached with a smile.
  14. Rational expectations – does the adviser think the dissertation is a perfect object to be carefully worked on over 20 years, or a project with fixed objectives that can be done in 2-3 years?

There are other desirable traits and remember that no adviser is perfect, but you need to choose someone who has at least a few very strong traits. Here are a few other good rules of thumb:

  • Super Star prof isn’t always the best adviser. Stars are often asked to go to a million conferences and serve on fancy committees. A lot of people don’t handle graduate education and these other tasks well, and grad students are often abandoned.
  • Ask around. If Grumpy Prof has been teaching for 25 years and has only placed one student, there’s probably a reason. Ask and you will find out. And don’t think you’ll succeed where others have failed – that’s what the other students said!
  • Appearances can be deceiving. Some folks may be great lecture hall instructors, but awful dissertation advisers, or vice versa. Once again, ask around.
  • Dig deep. Is the great placement record of Prof X’s students dumb luck? Did people succeed despite the awful behavior of Prof X? Get a sense of how Prof X helped out.
  • Avoid junior faculty. In general, most junior faculty are still figuring out the academic game. Also, they tend to move around a bit, especially if they are hot (in the academic sense). Exception: In fast moving technical field, like computer science, a junior adviser may be the *only* person who is on top of things.
  • Don’t use stereotypes. Just because Prof X is of the same gender/race/political persuasion/etc as you, it doesn’t mean they will be a good pick. Don’t let these sorts of characteristics blind you to their weaknesses. What matters is that they can help you become the scholar you want to be. And remember, if you read closely, I said the adviser has to tolerate people like you, not actually be like you. As long as someone can be tolerant, they can usually have a strong work relationship with some one who is very different than themselves.
  • Be prepared for rejection. Some good profs may be overloaded with students, too close to retirement or may not like you. So if you ask to work with them, you might get rejected. It’s ok. Just ask someone else.

Finally, have reasonable expectations. Faculty members are just middle aged men and women with their own careers and families. Don’t expect miracles, but if you do your homework and ask around, you will probably find someone who is professional and helpful.

Written by fabiorojas

April 23, 2007 at 2:31 am

the intellectual

Teppo

Mark Oppenheimer has a nice essay in the Chronicle Review on the lack of broader intellectual curiosity by graduate students (and professors for that matter), specifically for intellectual and public matters outside their narrow domains.  I in part concur with his proposed solution:

I have long believed that admissions committees at graduate schools should work very differently. Instead of asking for letters of recommendation from undergraduate thesis advisers, admissions committees should try to figure out if an applicant is an intellectual. They should ask: “What do you read outside your proposed field of study? What are your favorite books? Where would you most like to travel, and why? What periodicals do you read?” If a student has no aspirations to travel, doesn’t seem to read much except within her undergraduate major, and shows no interest in academic debates — well, that’s a bad candidate for academe. A bright, kind, loyal person, perhaps, who could be a success in many ways. But a bad candidate for the academy that America needs.

Now, that might be a bit over-the-top, lots of very narrow folks have obviously come up with brilliant theories and innovations, though, in principle his point resonates.

Written by teppo

September 24, 2006 at 5:16 am

Posted in education, research, teppo

student evaluations? sad!

A few weeks ago, I wrote a post about how I changed my mind about student evaluations of teachers. Early evidence showed that they correlated with student learning, but newer analysis reverses this conclusion. The James G. Martin Center, with my permission, edited and reprinted the post. A teaser:

The fundamental issue is that colleges should probably not be judged using the logic of consumer satisfaction. The metric of customer happiness makes sense for products that are meant to be immediately entertaining, like watching television. But education, especially higher education, is about learning which, by nature, is a stressful and inconvenient thing. And the onus for success should be at least as much on the student as on the teacher.

Check it out!

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome!

Written by fabiorojas

March 1, 2018 at 5:53 am

academics of the past — they’re just like us!

So last week, for me, was the first week of classes. But the week before that, I was in Austin to use the LBJ archives.

The LBJ archives are awesome. Of the 15 or so archives I’ve visited over the last 10+ years, they had hands-down the most helpful archivist I’ve ever met. (Allen Fisher, if you’re taking notes.) I was also amused that they’re a little competitive with the other presidential archives. I received a meaningful look when they learned that no one at JFK explained the details of the archives’ shared cross-referencing system to me. (“You’d be surprised how often that happens.”)

2013-10-01_LBJ_Library_Jonathan.Garza876

LBJ didn’t believe in windows.

Anyway, the best find of the trip was the papers of Donald Turner, who was the first economist to run the Antitrust Division (1965-68). There were letters to all sorts of major players in law & economics, like Robert Bork. A letter of recommendation for Oliver Williamson, who very early in his career was Turner’s special economic assistant. A “P.S” on a letter from the Dean of Yale’s Law School: “Is Steve Breyer [then 27] as able as I think he is?”

But the most fun was the totally human part. The year before Turner became antitrust chief, he went on sabbatical at Stanford, where, as academics do, he rented the house of another faculty member who was also on leave — a young Bill Baxter, who would become Reagan’s antitrust chief some 15 years later. Turner kept a copy of his outgoing letters from that year, and they sound awfully familiar. Academics of the past — they’re just like us!

They complain about grading!

I’m not sure if these were left over from the fall semester at Harvard, or if Turner had to teach while he was on leave, but the 140 blue books staring him down in December caused a fair bit of grief. That one’s easy to relate to.

They complain about how their articles are going!

In 1965, Turner published a major article on conglomerate mergers, which caused him all sorts of anguish. The letters are full of agonizing over how hard he’s working on it, frustration with how long it’s taking, and the inevitable requests for deadline extensions. To top it off, there was a final kerfuffle over the copyediting. He sure must have been glad to see the back side of that one.

They take advantage of the bar!

Turner and Baxter, the two future antitrust chiefs, didn’t know each other well at the time that Turner rented Baxter’s house. In fact, it appears that they had never met in person. One of the funniest bits is the post-sabbatical correspondence tidying up the loose ends. Turner drank some of Baxter’s booze, and offered to reimburse him for it. Baxter had noted what they left on a card, but then misplaced the card. So he roughed it out: “[My] recollection is that you left about the same amount of gin and wine that we had left; but that about two fifths of bourbon, two fifths of scotch, and one fifth of cognac have not been replaced.” He called it $25, and Turner paid up. It’s always important to pay for your liquor.

Written by epopp

January 30, 2015 at 1:30 pm

Posted in academia, economics

the indiana way of graduate education

I am often asked: how does Indiana consistently place people so well? Just to be clear, in terms of elite placement, we do as well as other programs of our rank. Our ranks floats in the 10-15 range, and we can do a good R1 placement every other year. Every few years we place at or near the top. What is amazing is that we consistently place all of our students, not just the “stars.” We place students that routinely get discarded at other programs. How does that happen?

First, it starts with the faculty. In hiring at both the junior and senior levels, we have a preference for people who are committed to graduate education. Also, in terms of culture, our faculty have a very different attitude towards graduate work. We don’t work on the “star system” where a few people get most of the attention. We believe that with the right training, we can help most graduate students start a career in either teaching or research. Finally, in terms of the faculty, we also tend to attract people who are productive and collaborative. Not much dead wood.

Second, we choose graduate students very carefully. Sure, we’ll hand out acceptance letters to a few “stars” of the market every year. That’s because, like most graduate programs, we take GPA and GRE scores very seriously.  But the difference is that Indiana doesn’t mark you down because you didn’t go to the “right” school. We’ve taken people from regional state schools, small liberal arts colleges, lower ranked MA programs, and other places. We read the entire application, not just the name of the BA school or the letter of recommendation. We’ve picked up some fantastic people who’ve gone on to great careers by looking for diamond in the rough.

Third, we have an insane amount of structure. It doesn’t work for everyone, but most people do well in a system that makes you jump through a lot of hoops. Like most soc programs, we have theory and methods in the first year. But we also have a summer research program, required writing workshops, a required minor, a required teaching workshop for TAs, and a bunch of other stuff. Annoying? Yes. But do people have the OLS model hammered into their brains at the end of the sequence? You bet.

Fourth, we have a deep “culture.” Our expectation is that all students should be able to complete the PhD. We also support students on multiple career tracks. We even have a special program for people interested in teaching intensive institutions. And nearly all faculty collaborate and believe in intensive 1 on 1 interaction. More than one visiting scholar has been shocked by this system.

Fifth, we have reasonable expectations and a high level of professionalism. We don’t pretend MA or PhD theses are master pieces. We want them to be good and we want them to be done. We also push people, but we (mostly!) don’t yell at people or treat grad students like children. We tell you what to do and then we try to help you. We’ll get back to you in a few weeks on your dissertation, not a few semesters.

Sixth is funding. While far from perfect, we’ve developed a system where most students can rely on 4-5 years of funding. Then, we have various mechanisms for helping most students. The pay is low, but we don’t play games. You get funding. You will get help. No games.

The issue with many programs is that they drop the ball on one or more of these issues. During my time at Chicago, for example, they had a nearly structureless program and very poor funding. Other programs will chase famous faculty without considering how well they place students. It’s too easy to slide into the star system of graduate training.  Considering all that, it’s a real testament to this department that we can go head to head with programs that have a fancier brand and a much bigger budget.

50+ chapters of grad skool advice goodness: From Black Power/Grad Skool Rulz

Written by fabiorojas

June 18, 2014 at 12:56 am

similiarities and differences among faculty by gender and partnered status

My last post on hiring and retaining faculty generated a spirited Q&A.  Brandy asked about differences by gender and partnered status.  Here are a few relevant snippets from the Stanford dual-couples report:

On partnered academics:

“– Women are more likely than men to have academic partners (40% of female faculty in our sample versus 34% of male faculty). In fact, rates of dual hiring are higher among women respondents than among men respondents (13% versus 7%). This means that couple hiring becomes a particularly relevant strategy for the recruitment and retention of female faculty.
– Women in academic couples report that their partner’s employment status and opportunities are important to their own career decisions. Not only do women more often than men perceive a loss in professional mobility as a result of their academic partnerships (54% for women versus 41% for men), but they actively refuse job offers if their partner cannot find a satisfactory position. In our study, the number-one reason women refused an outside offer was because their academic partners were not offered appropriate employment at the new location. These findings have significant implications for institutions seeking to recruit top women.”

On stay-at-home partners:

“Thirteen percent of our survey respondents have partners who are not active in the paid labor force. Men and women have very different partnering patterns in this regard (Figure 2). Most striking is that 86 percent of academics with stay-athome partners are men. These men face particular trade-offs in their careers. On the one hand, they generally have someone who manages the household. This can be tremendously helpful. They also tend to be more mobile. Even though families, especially those with children, do not like uprooting and making a new life for themselves in a new community, they often do. On the other hand, these families must survive on one salary.

Read the rest of this entry »

Written by katherinechen

August 23, 2012 at 7:13 pm

Posted in academia

productively handling structured and unstructured time as a scholar

In this post, I want to follow-up on my previous posts about conducting research by discussing the thorny issue of time management.  One challenge of academia involves completing work under schedules that incorporate both structured and unstructured time, with both unclear ends (what does one want/need to accomplish?) and means of reaching these ends (how does one achieve that goal?).  People must learn to self-manage the processes of undertaking a dissertation or research project and preparing publications along with other responsibilities.

During the school year, class preparation and grading, committee work (i.e., admissions committee, curriculum committee, hiring committee, etc.), service to the profession (i.e., reviewing manuscripts or conference papers, committee work for professional associations, etc.), and other commitments structure schedules.  For some, research, writing, and publishing all get squeezed into the remaining time.  Thus, periods such as the summer, winter break, and sabbaticals usually start with a long list of best intentions of how to spend “unstructured” time, which can feel overwhelming.

What to do?  This post is devoted to examining several Jedi tricks that increase the likelihood of accomplishing research projects during both structured and unstructured times.

Use deadlines

As part of class requirements, I used to assign my students two two-page long journal entries to help them understand the link between theory and phenomena (say, how routines help direct workers but may have undesired consequences).  The deadlines for these assignments were open-ended as students could pick whichever readings they wanted to use to analyze their organizational experience.  The assignment was due the same day that the reading was due.

Although a few students submitted their work on time, most students struggled with selecting their own deadlines and waited until the semester’s last week to turn in their journal entries.  A few didn’t submit any entries at all.  After a couple semesters, I tried another tactic: I made the first of the two journal entries due by the semester’s midpoint.  Student turn-in rates improved during the first half, but students still had problems with turning in the second journal entry.  After reading behavioral economist Dan Ariely’s description of his experiments with having MBA students set their own deadlines for turning in assignments versus setting deadlines for them (the MBA students at his elite institution apparently did no better than my undergrads in setting their own deadlines), I finally replaced this requirement with regular homework assignments with set deadlines.

Such experimentation shows how setting deadlines can be helpful, even if the deadlines are arbitrarily imposed.

How to set deadline prods:

– Understand prioritization

For scholars, juggling multiple responsibilities often means that projects that lack hard deadlines or immediate reinforcement can fall by the wayside.  It’s too easy to prioritize not particularly important deadlines for obligatory service commitments or bureaucratic paperwork simply because these have set deadlines while other, often more important or consequential responsibilities do not.  Or, some may find that the instantaneous gratification of teaching or mentoring students can trump the often-lonely, seemingly thankless task of writing up research and responding to reviewers’ comments.   These “pulls” can derail research productivity, particularly during long projects where deadlines are self-set – for example, submitting journal or book manuscripts for peer review.

For those of you who enjoy making 2 by 2 typologies, time management guru Stephen Covey suggests writing down projects and responsibilities in an important vs. not important and urgent vs. not urgent table to assess how you are allocating time.

– Routinize large projects into small incremental tasks

Based on his research comparing the writing and publication productivity of faculty who wrote in spurts versus regular, steady writing, Richard Boice recommends setting up small, incremental deadlines.  Rather than “binging” on intermittent writing bouts, he suggests regularly writing small amounts.  Some people assume that they have to be motivated first in order to be productive – instead, productivity is like a waterwheel: productivity elicits productivity.

– Follow a preset template

Wendy Belcher published an excellent guide to how to submit and publish journal articles according to a set schedule.  Her chapters cover topics such as how to write letters of inquiry to editors and how to respond to reviewers’ comments.

– Use other external deadlines

Some scholars find that presenting at conferences helps with getting initial drafts done, with the possible bonus of getting useful feedback during the review process or presentation.  If you’re writing journal manuscripts, check out calls for special issues, which usually have hard deadlines.  These also have the added advantage that reviewers have to get back to submitters by a set date.

– Participate in a writing group or colloquiums

Writing groups or colloquiums where members regularly present drafts for feedback can be great prods for productivity.  Depending on what your needs are, writing groups need not include only members from your own discipline – often, those from other disciplines can offer writing feedback that extend beyond substantive content, or they can suggest alternative perspectives which can be very helpful for cross-fertilizing with other fields.  Your university might even have a program led by a trained facilitator who will set up guidelines for a group.

– Intermix different types of deadlines

Sometimes deadlines for smaller projects can feed larger project deadlines by supporting substantive knowledge.  For example, if you are asked to write a dictionary entry or review a book related to your research topic, you now have the opportunity to distill your knowledge of existing literature.  Successful submission can set up conditions for entrainment – that is, meeting a small deadline might provide the impetus to pursue a larger deadline.  The tricky balance is not to take too many small projects at the expense of a larger one with a bigger impact.

– Use carrots and sticks

To meet deadlines, some colleagues have used carrots like a non-refundable vacation or moving to a new job.   A stick might be running out of funding – a “natural” end to a project.

– Work with collaborators

If you’re the type of scholar who prefers company, you might find that the stimulation of working interdependently with others is more appealing than working independently.  However, this can be a double-edged sword if the collaborators (or you) are overly optimistic about abilities to expend time.   Most likely these will involve frank conversations about authorship and responsibilities upfront, as well as adjustments along the way.

– Spend regular time with friends and family; participate in a hobby

Finally, some might feel tempted to eschew “distractions” until a big project is over.  However, scheduling in hobbies and regular downtime with friends and family – even a deceptively mundane task such as a walk with a pet – can help motivate scholars over the productivity hump.

Other resources

Randy Pausch’s recorded talk on time management

A handy guide to how research progresses (or not)

Add your recommendations to the comments…

Written by katherinechen

August 2, 2012 at 3:05 pm

another job

I’ve been involved with an interdisciplinary special issue on collective decision-making and behavior.  One of the scholars contributing is David Sumpter, a mathematician who studies the collective decision-making and behavior of animals.  If you are interested in the topic, check out David’s book (Princeton) Collective Animal Behavior.  It’s fantastic.

So, since we’re advertising jobs around here (see Brayden’s post about Northwestern) – David mentioned that he is looking to hire a postdoc for his research group to study human collective behavior – a great opportunity.  I think there are some really novel insights to be had in studying collective, organizational behavior and interaction across contexts, in comparative fashion, focusing on both similarities and differences.  This seems like a cool opportunity.

If you are interested, more details about the position here or below the fold. Read the rest of this entry »

Written by teppo

November 4, 2011 at 9:02 pm

Posted in academia, uncategorized

tyler cowen’s true, but brutal, job search advice

From Marginal Revolution:

Every now and then I give informal talks on how the economics job market operates.  I tell the listeners that they are like an “unwrapped saltine cracker.”  They are wasting assets, to borrow a phrase from options pricing theory.  If a day goes by and they did not accomplish something important, they decline in value.  For most candidates, holding steady is not a viable strategy.  You need either publications or some stellar letters from credible writers, preferably both.  (At the very top level, publications at the job market stage are less important because it is expected they will come and the recommendations are trusted more.)

Unwrap a saltine cracker, let it sit for months, and then try to eat it.  Will you even try?

I did a similar talk for some graduate students at a good, but not high ranked program, and they asked about what students in their position should do. My new response: read this paragraph – and get back to work!

Written by fabiorojas

June 26, 2011 at 2:40 am

Posted in academia, fabio

the editors speak: what makes a good review

One of the most important things we do as members of an intellectual community is assist in peer review.  As important as it is, reviewing papers is one of the tasks that receives the least amount of attention in graduate school training.  We certainly learn how to critique in grad school, but, as you’ll see by the editors’ comments below, critiquing is not the same thing as reviewing. Most of us learn how to be reviewers simply by doing it.  While there will never be a definitive how-to manual for reviewing, I thought it would be nice if our field could identify some of the best practices in reviewing. With that idea in mind, I asked a number of current and former editors at journals in organizational theory and sociology to comment about what they think makes a good review.  This post includes their thoughts.

You’ll notice that the editors seem to agree on several important points (e.g., be constructive!), but there is some variation as well.  Some of the editors make very specific and useful points about what reviewers should and should not be recommending in their reviews. Rather than summarize, I’ll just let you read it for yourself. I’ve put their responses in no particular order.

Read the rest of this entry »

Written by brayden king

May 31, 2011 at 3:13 pm

Posted in academia, brayden, research