orgtheory.net

how i pick grad students

Don’t take this post personally – I haven’t been on a graduate admissions committee in a while. But I wanted to start a thread on how sociology programs pick doctoral students.

Things that impress me:

  1. Overall excellence: Like nearly anyone, I am impressed with applicants who have a high GRE score and GPA.
  2. Courses that are tough: I am impressed when people do well in courses that are regarded as demanding. In the sociology major, these would include social theory and statistics. Outside the sociology major, I am impressed by foreign languages, philosophy, and anything that requires non-trivial amounts of mathematics. I also give slack for the challenge. The B+ in advanced Arabic is probably more persuasive to me than an A in intro soc. For this reason, I read transcripts very carefully.

Things that sometimes impress me:

  1. Research papers/writing samples: If they are well written and address non-trivial topics, then I impressed.
  2. Choice of major: If you can maintain a decent, though not perfect, GPA in a really challenging major, I am impressed. I am not impressed by bad performance in a tough major. I also expect near perfection from applicants in notoriously easy majors.

Things that usually don’t impress me:

  1. Statement of purpose: I usually find this useful only as a test of basic language skills. It also communicates that the applicant is a good fit with respect to research interests. But these considerations rarely come into play because I can usually tell what I need to know from your transcript and test scores.
  2. Letters of recommendation: As I have argued many times on this blog, most research, with a few exceptions, shows that letters of reference are poor predictors of future performance. I quickly scan letters for red flags to weed applicants who are clearly not suited for graduate school. But aside from that, I pay very little attention to them.
  3. The college you came from, unless it is known as an incredibly demanding/undemanding place.

Stuff that I ignore:

  1. CV’s/resumes
  2. Personal stories/life experience
  3. Work experience, unless it is obviously related to your research and it’s really unique.
  4. Internship/Summer research experience. Nothing wrong with those activities, but I don’t have any personal experience or research showing me that these predict success.

I know that people disagree with me on many points. I make no claim that other faculty use these criteria. I’d be interested in your opinions with regard to evluation of graduate school applicants.

Written by fabiorojas

May 6, 2011 at 12:15 am

Posted in academia, fabio

63 Responses

Subscribe to comments with RSS.

  1. I’ve always wondered how sociology graduate programs regard transfer applicants, or students who have completed graduate work in other departments. I’ve heard that in the hard sciences there is very little stigma attached to transfers, while in the humanities there’s usually more drama involved. Is sociology somewhere in between?

    Like

    anonymous

    May 6, 2011 at 1:40 am

  2. I don’t have data, but my gut feeling is that you are right. It’s in the middle for sociology. I was a cross-over, as are many people I know. But sociology can be a very traditional discipline as well and some departments few cross overs.

    Like

    fabiorojas

    May 6, 2011 at 1:56 am

  3. As a grad student not so far along that I don’t remember the agony of application season, I find this very interesting.

    Question: How do you evaluate non-traditional college students? The criteria listed above would seem to put students without parental aid (like most non-trad students) at a serious disadvantage. If you ignore personal details, how do you know the difference between a kid taking it easy with 3 classes/semester, and one working two jobs to put herself through college? Comparing the transcripts of these two students as if they’re the same seems flawed.

    Like

    Chad

    May 6, 2011 at 3:16 am

  4. Chad: That’s a good question. And I will readily admit that the grad school application process favors students from comfortable backgrounds and traditional transcripts.

    My overall strategy is “signals that can’t be faked.” I am very wary of letting personal information affect my decision. Academia is a very tough business and I don’t want to encourage someone who has a very small chance of success.

    You asked: “If you ignore personal details, how do you know the difference between a kid taking it easy with 3 classes/semester, and one working two jobs to put herself through college? ”

    My answer: I look at the actual courses. Taking it easy probably doesn’t mean taking statistics or some other challenging courses. The working student could signal that they had the right stuff by taking fewer, but equally challenging courses.

    Like

    fabiorojas

    May 6, 2011 at 3:38 am

  5. I am not quite in agreement with Fabio. In my 20 years on grad admissions committees, I have used many heuristics and only a few are universal across individual’s dossiers. GRE and GMAT scores are marginally valid and not particularly good predictors of success. Where the applicant studied matters, especially if it is a reputable liberal arts college and especially if they majored or minored in biology or chemistry. Big pluses. The statement of purpose matters. Big time. I will vote against your admission if you don’t know or are unspecific about what you wish to specialize in; I don’t want you to be admitted without the propect of a good fit with a research advisor. If our department faculty and ongoing research programs don’t fit with you, you shouldn’t be admitted.

    Your transcript matters. I look to see which courses you dropped and took incompletes

    Like

    Randy

    May 6, 2011 at 4:18 am

  6. Randy: How do you arrive at these rules of thumb? Does your program have detailed evidence on what actually predicts completion of the program?

    Like

    fabiorojas

    May 6, 2011 at 4:32 am

  7. [took incomplete grades in]. How soon in your prior degree did you stop filling the semesters with low-level courses? Did you do any independent research?

    Like

    Randy

    May 6, 2011 at 4:33 am

  8. Interesting. Somehow it didn’t occur to me that details like personal background would be faked, but I guess it would be pretty easy for anyone to claim some kind of hardship to justify a checkered academic history. And it sounds about right that course selection is telling, no matter how many courses a student takes in a semester.

    So it makes sense to put weight on these signals that can’t be faked, but what about those that can be faked, but could also be verified with relative ease? After all, it isn’t *that* hard to check on an applicant’s work history. Employers do this all the time, and no doubt it could be done for the relatively small number of applicants where it would make a difference, if departments wanted to check.

    I guess my point is that it would be nice to see the application process give non-traditional students a fair shake–or at least to know they’re not being penalized for circumstances that have nothing to do with their ability to succeed in academia. Including a broader range of considerations is messy and might require more work (and having not been on an admissions committee, I admittedly have no idea how much work this already is). But if departments value a diversity of students, it seems like they should find a way to take some of these other factors more seriously.

    Of course, I also have no idea how non-traditional students fare in grad school compared to others. Anecdotal evidence suggests they’re doing just fine (so far, at least!), but in the aggregate, I can imagine reasons why it would go either way.

    Like

    Chad

    May 6, 2011 at 5:30 am

  9. ” I also expect near perfection from applicants in notoriously easy majors.”

    Such as?

    (For the record, I have degrees in Sociology, Demography and Applied Statistics. This is not a personal question, I’m just curious).

    Like

    Guillermo

    May 6, 2011 at 5:54 am

  10. Prof. Rojas, I think that it would be fair for you to also give your impression of what professors in general, or what other professors you know, seem to regard as “impressive.”

    Like

    Andrew

    May 6, 2011 at 7:48 am

  11. Do admissions consider only undergrad grades? I’ve been thinking about phd, but honestly I didn’t try very hard in my undergrad (didn’t think of academia at the time) so I’m wondering if I can strengthen my app with a much better performance via a masters first.

    Liked by 1 person

    Chris

    May 6, 2011 at 12:07 pm

  12. My sentiments are similar to Chris’s, except that I am in the middle of a Master of Arts in Sociology at a school that does not offer a PhD in Sociology. In your estimation, to what extent will stellar performance in the Master of Arts program supersede slightly above average performance at the undergraduate level?

    I am especially concerned about the apparent difference in grading standards from undergraduate- to graduate-level courses. For those of us wishing to get into a decent PhD program, earning anything short of all As is considered a serious problem (as expressed by my graduate director). But for students applying to a PhD program directly out of undergrad, the standards seem to be a bit more relaxed and comprehensive.

    I know that if Dr. Andrew Perrin’s idea of including each course’s median grade on student transcripts were applied at the graduate level, my As would look much less impressive next to all those median As.

    Like

    Pretendous

    May 6, 2011 at 12:34 pm

  13. My central experience having served on admissions committee is surprise at the diversity of what people choose to focus on. Absolutely, there are plenty who take the statement of purpose, letters of recommendation, work experience, and (especially) where people went to college quite seriously, and that many people do not read transcripts that closely in terms of the content of courses (although they do look at overall summary GPA). Some people really like sociology majors; some people have a disciplinary-self-loathing thing where they really like majors typically more difficult than sociology.

    Upshot: if you are applying to grad school, your application will be judged by a committee, so any one person’s reflection about what impresses them should be put in context. Fabio doesn’t present his post as anything more than what he personally uses as criteria, and his should not be taken as necessarily typical–in my view and experience, they seem fairly unusual in their rank order after #1.

    Like

    Jeremy

    May 6, 2011 at 2:32 pm

  14. Ditto to what Jeremy said. There is a lot of randomness to the admissions process in part because there is so much heterogeneity in what people focus on. That said, given the heterogeneity problem, I think you’re right to focus on improving your GRE score. People with really high GRE scores tend to get evaluated more carefully. You’re probably going to make the short list if you’re scores are in the 90+ percentiles.

    Like

    brayden king

    May 6, 2011 at 2:42 pm

  15. But why not say the truth? That entrance into a graduate admissions program is determined by promising to work with a high status professor in the department, in the lines of research s/he is interested in? All this talk of GPAs and GREs and the like is company line bullcrap (I like you guys nonetheless).

    Like

    Guillermo

    May 6, 2011 at 3:49 pm

  16. Guillermo – that’s just not true. As my comment and Jeremy’s above show, there are lots of reasons that students get admitted. It’s a complicated process and it may vary from year to year. I think GPA and GRE end up mattering quite a lot, especially as initial screening devices.

    Like

    brayden king

    May 6, 2011 at 4:09 pm

  17. That entrance into a graduate admissions program is determined by promising to work with a high status professor in the department, in the lines of research s/he is interested in?

    No. See above re: credible signals.

    Like

    Kieran

    May 6, 2011 at 4:17 pm

  18. Lol. I’ll apply to Duke and Northwestern then.

    Like

    Guillermo

    May 6, 2011 at 4:47 pm

  19. I personally like to run admissions by “beaming” the applicants to a desert planet that has bamboo, diamonds, and the ingredients for gunpowder. Whoever wins the trial by combat gets a tuition waiver and if they are merciful enough to refrain from a coup de grace they also get a very generous stipend package.

    Liked by 1 person

    gabrielrossman

    May 6, 2011 at 4:54 pm

  20. Sadly, my distaste for interstellar space transport and an acute allergic reaction to potassium nitrate kept me from ever applying to UCLA.

    Alright, so here’s another question for everyone: To what extent do you get the sense that individuals on admissions committees end up supporting applicants that resemble themselves? I have a vague recollection of some research along these lines in hiring employees, and have certainly been on (non-academic) hiring committees where this was quite obviously the case. It doesn’t seem like such a stretch to imagine it in graduate admissions.

    GPA and GRE are obviously credible signals, but it seems that other potentially useful information is likely discarded because committee members simply don’t know what to make of it. In employees searches, I had no concrete sense of what college athletics or fraternity membership brought to the table, and I’ll admit that I more-or-less ignored this info. Others who could identify with these experiences made cases for their value, but such cases are argued only if someone on the committee can relate. Depending on the makeup of the committee, that leaves a lot of people out in the cold.

    Of course, if admissions committees are a fair reflection of the department as a whole, then the committee’s thoughts/experiences on these factors are probably a fair indicator of some less tangible aspects of an applicant’s “fit.” But that only holds if there’s variation between schools on these less universal criteria. (My sense is that this is true. I’m very happy in my program, but can imagine being pretty unhappy at some very good programs that rejected me.)

    I’m sure that the answer to my initial question is that “it depends,” but I’m curious about your thoughts on this.

    Like

    Chad

    May 6, 2011 at 5:18 pm

  21. Chad: If your question is along the lines of whether faculty who went to elite undergraduate institutions are more likely to give that more weight, or if faculty who majored in disciplines other than sociology have a relative preference for non-sociology majors, or if faculty who took time off between undergrad and grad school are relatively more congenial to applicants who did the same thing, etc., my anecdotal sense is consistent with this. I would further conjecture these patterns are stronger for grad students on admissions committees than faculty, and stronger for younger faculty than older faculty. It’d be interesting to see a study of it.

    Like

    Jeremy

    May 6, 2011 at 5:33 pm

  22. Chad,

    Honestly, on the homophily thing what comes up more often is admissions trying to figure out if there would be somebody who would be willing to work with an admitted student. For instance last year I wasn’t on admissions but the committee forwarded a few files to me soliciting my opinion and commitment to work with them if they were admitted.

    My judgement in these cases has basically nothing to do with the personal statement name-dropping me. Usually a sentence at the end of the personal statement that “i am excited to work with Rossman” means that the applicant saw the word “culture” in my faculty profile and concluded that I do some kind of cultural studies or media effects. I have nothing to offer students who are interested in questions like “how are our attitudes about X shaped by media portrayals.” Rather I’m much more interested in whether the substantive parts of the personal statement and writing sample reflect parallels to my theoretical interests in econ soc and production of culture.

    Also, I agree with Fabio and Brayden that GRE matters a lot. I’m also very interested in the writing sample, especially if it reflects original research of some kind. Finally, I want to see evidence in the statement and sample that the person has a deep interest in developing theory rather than having the misimpression (as is encouraged by our undergrad pedagogy) that sociology is entirely about social justice advocacy.

    Like

    gabrielrossman

    May 6, 2011 at 5:49 pm

  23. There’s been a lot of useful discussion. Let me add a few comments:

    @Chad: I mean “fake” in two senses. First, people may not tell the truth about their hardships. But, honestly, that’s rare. Second, I mean “fake” as in “this activity doesn’t necessarily indicate academic skill.” For example, you may talk about study abroad in your application. That may indicate real talent, or it may simply indicate that your folks have disposable income. It’s a signal that can be faked. Or in econ talk, it’s not a credible signal.

    @Guillermo: I would back off a little on your comments. Try working with low GPA and GRE students. It ain’t pretty. That’s why we take GPA/GRE seriously. Also, faculty fit is very important. Yes, some professors do treat admissions as a way to get lab workers. But you can’t ignore fit. For example, if you really want to do social psychology, you can’t go to my alma mater (Chicago) because they simply haven’t trained in that area in a long, long time. Similarly, don’t come to Indiana to do urban studies. You need a lot of faculty help to get through a PhD program.

    @Jeremy and Brayden: Yes, I did present my list as a personal opinion. But some of my views are more defensible than mere opinion. For example, very little job performance research shows that letters of reference predict success. This is a well known result in psychology and it would simply be irresponsible of me to take letters seriously given the state of research on the subject.

    Liked by 1 person

    fabiorojas

    May 6, 2011 at 6:23 pm

  24. Fabio, with respect to my heuristics, they are developed through experience and are thoroughly filtered by my personality, peculiarities, and perspectives. My department and all the others in which I have worked eschew collecting data on input-output. I believe that the greatest predictor of success in the PhD is finding an early and fruitful relationship with a mentor/advisor. Students who are unlikely to attract attention of good mentors wander in the wilderness during courses, take forever to complete a proposal, and end up leaving before completion because the rice bowl is empty.

    Most of the comments above are correct: it is a committee process and multiple perspectives are in play. Some people care about GREs. But most applicants fall into the middle category — “who cares”. They are acceptable but no more. But any committee member can champion such an applicant because they have a certain attribute. I’ll champion one or two a year. One recent case was someone who was a published poet as an undergrad. And any committee member can torpedo any applicant because of a single negative attribute, because nobody wishes to expend the time, emotion, or effort to fight for an adequate student if someone is fighting against.

    Liked by 1 person

    Randy

    May 6, 2011 at 6:35 pm

  25. Randy: Are you really telling me that you “eschew collecting data on input-output?” Really? That blows my mind. I would welcome decent research on student success.

    Like

    fabiorojas

    May 6, 2011 at 6:42 pm

  26. Fabio – I agree with you that letters are mostly unhelpful (unless they say anything close to negative, then I’m all ears).

    Liked by 1 person

    brayden king

    May 6, 2011 at 7:05 pm

  27. This report on GRE scores is kind of old, but I’ll just pit it here: http://www.news.cornell.edu/releases/Aug97/GRE.study.ssl.html

    It appears that GRE scores are not a very solid indicator of success in graduate school.

    Like

    coqui21

    May 6, 2011 at 7:05 pm

  28. Coqui21 – the problem with that study is that they’re only looking at students who were accepted into an elite graduate program. It didn’t track the success of graduate students who were not accepted into that program but made it into other schools. And of course it didn’t track the success of students whose GRE scores were so low that they didn’t make it into any graduate program and who were, by definition, unsuccessful academics.

    So yeah, I completely buy the idea that once students have been sorted into the various graduate programs that GRE will not be a very good predictor of success, but I think that GRE is a very useful signal of quality in the first round of sorting into grad schools.

    Liked by 1 person

    brayden king

    May 6, 2011 at 7:17 pm

  29. So, I’ll be the jerk and ask What are we defining as success? Do we mean graduating with a PhD? Do we mean getting a tenure track position at an R1?

    Also lets use a bit more active construction of Fabio’s statement that “the grad school application process favors students from comfortable backgrounds and traditional transcripts”. “Grad School” isn’t doing something, we are. Or in this case, Fabio is.

    Like

    Fanon

    May 6, 2011 at 7:27 pm

  30. fanon,

    i’m open-minded about what sorts of placement constitute success but it’s ridiculous to even question completion as part of success. it neither benefits the program nor the individual for someone to spend 10 years or so in grad school and never complete a dissertation.

    Like

    gabrielrossman

    May 6, 2011 at 7:53 pm

  31. I was not suggesting that completing the degree is a good measure of success, nor R1 placements. I just want specificity, which is hardly a ridiculous request.

    Like

    Fanon

    May 6, 2011 at 8:17 pm

  32. BK- Good point about the study. I too think the GRE is probably a fair (though not great, nor encompassing) indicator of ability in the first rounds of sorting. More than anything else, I think it makes it easier for admissions committees to weed down their long lists of applicants without much effort.

    Like

    coqui21

    May 6, 2011 at 8:36 pm

  33. I put less weight on GREs than a lot of people. I agree they are highly correlated with ability. BUT: (1) they are also highly correlated with class, especially parents’ education. (2) High GREs + mediocre undergrad record more often than not turns out to be someone who just never gets it together as an academic. Typically the person comes from an educationally advantaged background. (3)Smart, scrappy disadvantaged people who end up doing really good work often have lower GREs than people from privileged backgrounds.

    I’m guessing most academics would agree on who the “sure things” are — the people who are obviously smart, obviously high-energy and creative who have already been mentored and have already exhibited the talent you are looking for in a grad student. But we disagree a lot about how to rank people who are not sure things.

    Like

    olderwoman

    May 6, 2011 at 8:44 pm

  34. I’m even less impressed with GREs as a predictor than olderwoman. I was graduate director here some years ago, and have watched each year’s entering class, noting that some of them leave with a degree, and that GPA and GRE don’t explain who those graduates are. (And I would consider reasonably timely graduation as someone employable as success.)
    The GRE data are, to be sure, censored on one end, so it is possible that someone who scores 300/300 could succeed in graduate school, but have no examples, because they’re not admitted. I’d be astonished if the difference between 600/600 and 800/800 GREs makes any difference in grad. school performance at all—even acknowledging that the student with the higher scores is likely to get a better funding package.

    Contrary to Fabio’s claim about how rough it is to work with low GRE students, I’ve very much enjoyed working with some students with GREs low enough that colleagues argued they were untrainable. Some of those students finished here in a timely fashion with multiple publications and are now having great careers. A program with enough applicants can choose only among students with high GREs, but it will still admit duds and will miss a lot of talent.
    So what matters? I’ve been paying attention long enough to have my initial biases (majoring in another field; writing well; personal statement) disproved. Personal resilience, drive, and the ability to embrace criticism are the best predictors of success, although it’s generally hard to judge those characteristics from the application. There are plenty of people who are smart enough, and only a few of those are sufficiently curious and courageous to push through to degree. (I find Dan Chambliss’s “mundanity of excellence” to be a very useful statement on all of this–although he’s writing about swimmers.)

    Like

    David S. Meyer

    May 6, 2011 at 9:03 pm

  35. “I am impressed with applicants who have a high GRE score and GPA.”

    Durkheim was second to last in grades in his ecole program. How many of those who scored higher than him can you name today?

    Like

    Guillermo

    May 6, 2011 at 9:46 pm

  36. I would love if every department other than mine made admissions decisions as if it believed there was actually zero ceteris-paribus difference between 600/600 and 800/800 GRE scores (a 600 math GRE is around the 50th percentile nowadays or a little below). I would encourage folks who think this to do their best to convince their colleagues accordingly — as a bonus, your department will significantly improve its yield rate by acting upon this view.

    Like

    Jeremy

    May 6, 2011 at 10:05 pm

  37. I agree with David Meyer that the most important predictors might be hard to assess using applications. Thus, it’s probably a good idea to not overly stress ‘objective’ measures like GRE and GPA.

    With GPA there’s obvious issues in privileging the applicants with high marks across the board. Difficulty of major and coursework is certainly one. But so is type of school–did they get higher grades at a school or in a field that emphasizes more rote learning over creative thinking?

    I would be so bold to argue that some degree of inconsistency (with a good proportion of high grades) is a mark of intellect. Those who are busy thinking about and working on the things their passionate for may not take the time to try to get an A in intro to whatever general education requirement they had to take.

    Like

    cwalken

    May 6, 2011 at 10:10 pm

  38. At my university, admitting a graduate student implies a six-figure financial committment, and we reject 8-9 students for every one we admit. It’s a task meriting the seriousness of a hiring decision, not casting a reality TV show. With all these criteria that people think it’s important not to pay much attention to, what exactly is the right thing to justify refusing one person admission so you can take someone else? Should we roll dice? Cast runes? Or go by whether we have a “good feeling” about someone? Whether they seem likable from their personal statement? Their hobbies?

    Like

    Jeremy

    May 6, 2011 at 10:35 pm

  39. “It’s a task meriting the seriousness of a hiring decision, not casting a reality TV show. ”

    I assure you that if your university has a fine arts or architecture program, they are going to *require* students to submit a portfolio of creative work, which is going to be a major factor in the application. If you are looking to train social scientists, I suppose you’re looking for scientific talent, not just meticulousness with previous homework and exams (which is what GPAs measure somewhat accurately). But if in this very same post you’re saying writing samples are requisites to be tossed aside…

    Like

    Guillermo

    May 6, 2011 at 10:44 pm

  40. I’d like to put in a plug for indicators of excellence that cross over or go against expectations. These include perfect GPAs at underdog schools, high math GREs for people coming to sociology from humanities majors, very good writing by stats-heavy applicants, etc. These are things that make me look more closely at transcripts and statements. Also, I do look at letters seriously, but very quickly. It would help if they were limited to 100 words, since that’s what it takes.

    Like

    Philip Cohen

    May 7, 2011 at 1:41 am

  41. i’m always baffled by the emphasis people put on the GRE, which is the same english and math as the SAT. what does it show you — that you took the gre a couple of years after you took the sat and remember some stuff? what about those who take it later in life? subject test is even worse — i went to binghamton undergrad so got a tremendous dose of world systems theory and just a smattering of other stuff. i would have done miserably.

    i’m also baffled by grades. yes you can look at transcripts closely. but graduate school is a different ballgame, not just in terms of the courses, but as a life form (am i using that term correctly? probably not. i never took orals or comps, another grad school hoop i never understood. ok, you’ve proven you can absorb a lot of info in a short period of time. good for you.) so doing well as an undergrad is no guarantee. especially those who go straight through. i saw so many people who had no work experience flame out. you just don’t know.

    i disagree with you on the letter. just from personal experience, i had horrific grades (seriously. i was premed for two years), even in my soc major. the one thing i did was read a lot. and think a lot. that showed through in my letter, which committee members told me led them to admit me in spite of bad grades.

    i guess if i were on such a committee, i would reverse engineer the process. we measure professional success by our written output first second and third, and teaching ability eighth. will gre scores really make you a good sociological thinker-writer? grades? where will you find that evidence, that little thing that really tells you hey, this person might make for a good sociologist? i think it’s in the personal statement. i doubt even in interviews if that would come through reliably, as so many of us are geeky and probably sweat too much for our own good. or too many are blowhards, good at talking, but little else. but the statement, that is good. unless it’s plagiarized. then…

    but since there are no jobs anymore, better to just shut your program down.

    Like

    syed ali

    May 7, 2011 at 1:03 pm

  42. I also wonder what the GRE measures. It’s a signal, for sure, but a signal of what? I have a degree in math and did abysmally on the math section. So while I could play nicely in non-Euclidean spaces, I couldn’t recall the formula for the area of right circular cylinder. I think the verbal section is more useful, but I’m not convinced how well definitional and logical familiarity with an array of words predicts success in graduate school. The GRE seems to be a decent signal of how hard one’s willing to work to jump through another hoop. Lots of folks study for weeks or months leading up to the GRE. Some even take pricey classes. But if all the GRE can tell you is how much someone was willing to relearn 7th grade math and ingest the Oxford English Dictionary, this is at best only a medium-run signal and not one indicative of long-run dedication to and success in one’s endeavors. The GRE measures how well someone can take the GRE.

    GPA has its own limitations. Maybe you spent the first two years of college languishing in a major you didn’t enjoy and finally mustered up the courage (or took a hard enough beating) to switch to something you found rewarding and, importantly, you’re pretty good at. It’s also hard to standardize across schools and majors. And doing well in classes might say little about one’s capacity to conduct interesting original research. But it’s a longer-run measure of achievement and signal of commitment and competence than the GRE. Surely repeated observations over time (GPA) are preferable to a one-off observation (GRE) if you think they’re measures of the same latent traits?

    Like

    Brian

    May 7, 2011 at 3:43 pm

  43. btw, about a year ago i had a post about how it makes a difference whether you read the score or percentile column of the GRE

    Like

    gabrielrossman

    May 7, 2011 at 8:01 pm

  44. fabiorojas: “Academia is a very tough business and I don’t want to encourage someone who has a very small chance of success.”

    I do not know any easy businesses. We all have talents, interests, incentives, and limitations.

    The assumption seems to be, Professor, that the applicant seeks a doctorate in order to continue in academia, to teach (perhaps) and to do research (certainly).

    My motivations are different and my achievements are beyond those of most people, as my cv – which you decline – demonstrates. For one thing, I am 61 years of age. My interest is to explore and report on fraud and misconduct in scientific research, a problem with a scarcity of criminologically rigorous studies. After completing my Ph.D., I would remain in the private sector, but I would have significant findings, and perhaps a book to my credit – and to the credit of my committee, of course.

    In short, I am shopping for a school that can meet my needs. After all, it is axiomatic in economics that trades are bilateral.

    Like

    Michael E. Marotta

    May 7, 2011 at 10:34 pm

  45. gabrielrossman replied to Fanon: “… it’s ridiculous to even question completion as part of success. it neither benefits the program nor the individual for someone to spend 10 years or so in grad school and never complete a dissertation.”

    Accepting the ten years as hyperbole, it is perfectly valid for someone to seek programmed, structured learning in a set of subjects, for a specific purpose that may have nothing to do with a degree. A person’s goal might be the creation of a series of statistical studies of academic misconduct, something which has not been done recently. Working with their peers – doctors and masters – they could then publish the results. They might be content with that much. Personally, that is not my choice, but it seems valid to me.

    I have taken college courses in Japanese, robotics, accounting, drafting, and computer programming. A post-graduate level of study for a specific purpose only seems to be professional development at a higher level.

    Like

    Michael E. Marotta

    May 7, 2011 at 10:47 pm

  46. Michael Marotta: The “it’s my choice” argument only applies if the student is paying 100% of the costs of that student’s instruction. Most of the discussion here involves programs that provide scarce assistantships to doctoral students. Departments are quite concerned about not “wasting” support and faculty energy on people who will not succeed which is, minimally, defined as completing a degree within some reasonable time span. Even if the hypothetical permanent students are paying full tuition, they occupy space and faculty time that could be taken up by someone else whose activities would also meet some of the department’s goals, not just the student’s.

    Like

    olderwoman

    May 8, 2011 at 1:21 am

  47. I had an interesting consideration:

    Say an applicant has been writing a blog with topics relevant to their discipline for roughly a year. Twenty percent or so critical examination of journal articles, the rest editorial about news headlines or redirects to other articles within discipline.

    Pros and cons of including mention to this on CV or personal statement?

    Like

    Mike

    May 8, 2011 at 5:34 pm

  48. Mike,

    If any faculty read the blog ask them to write a letter of rec.

    Like

    gabrielrossman

    May 8, 2011 at 6:51 pm

  49. @Guillermo
    In Durkheim’s class, there was Henri Bergson (philosopher and Nobel prize of literature , http://en.wikipedia.org/wiki/Henri_Bergson – finished 2nd) and Jean Jaurès (left-wing political leader and moral authority, politically assassinated when WW1 started http://en.wikipedia.org/wiki/Jean_Jaur%C3%A8s – finished 3rd).

    Obviously there was no great sociologist in Durkheim’s class, since Durkheim founded French sociology.

    Like

    Johnson

    May 9, 2011 at 8:00 am

  50. I would begin with the question: what are key success factors for a PhD student? I doubt we could reach consensus but I think, after some basic level of intellectual horsepower, the top 3 are something like: 1) creative problem-solving, 2) tolerance for ambiguity, and 3) writing skills.

    Despite all the focus on quant skills, I’ve never known a PhD student to fail because he/she could not learn regression analysis. Probably the biggest problem is that other educational programs are much more structured so people fail because they cannot figure out how to keep moving forward in the context of great ambiguity. They can’t organize themselves.

    Unfortunately, the standard application materials are not geared toward these 3 criteria. The analytical writing score helps show if the person can construct a logical argument. We are left with the essay for a test of creativity. No real indication of tolerance for ambiguity — though sometimes there is a hint in their experience.

    There is a significant return to the school that develops better signals of these factors since PhD programs are an expensive use of resources…

    Like

    RussCoff

    May 9, 2011 at 11:01 am

  51. @Brian re: “The GRE seems to be a decent signal of how hard one’s willing to work to jump through another hoop…The GRE measures how well someone can take the GRE.”

    Agreed on both counts, but it seems like the general conclusion you’re drawing is that because of this it’s not a particularly useful signal (feel free to correct me if I’m misinterpreting). Willingness to work hard and jump through the hoops necessary for one to pursue the career their passionate about is, I think, an incredibly important thing, and a good check on whether or not an applicant is serious. It’s not a sufficient indicator to tell whether or not someone will be a successful researcher. Not everyone who is willing to jump through the hoops and work hard will be successful, but far more people who aren’t willing to do those things will fail.

    From an intellectual perspective, the GRE does measure how well someone can take the GRE. And that’s not particularly valuable. But from a motivational perspective it signals how seriously someone takes their chosen career path and is willing to do the small things along the way to be successful, and that’s important.

    Like

    Sam

    May 9, 2011 at 3:34 pm

  52. I’m not entirely sure why the GRE is considered a good indicator of success…Students who do well on the GRE are more likely to get into graduate school than those who do not. Thats all we can say. To say that those who do well on the GRE are more likely to be successful in their graduate program than those who do worse is tautology. Of course you’ll see higher performance from those with high GRE scores – they got into graduate school!

    Sure, someone going to LSU may have a 1200, while someone attending Harvard might have a 1400+. I hardly think this difference can account for higher performance. You really need to control for things like advisor prestige and the social capital that comes with an Ivy league admission if you’re going to make comparisons between high and low GRE scores – which is really a qualitative endeavor.

    I think grades are a strong indicator of success. Students who seek a graduate degree are not “taking it easy”. Graduate students must work hard in order to achieve – a solid undergraduate GPA will indicate that. You won’t see a slacker applying to graduate school…I laugh when I hear arguments that claim “this student went to a less prestigious school – their professors probably curved grades and gave the majority of the class high marks.”. Slackers aren’t receiving high marks – whether they attend a prestigious school or a lower ranked school.

    Strong GPA’s indicate serious aspirations. Strong GRE’s might indicate….an ability to memorize data and methods? I mean…you have to study in a particular manner for the test. It’s not a general knowledge test by any means. You have books that aim to “crack the GRE” and courses taught by ETS that promise to raise your score by 200 points. If the GRE was a general knowledge test you wouldn’t see this.

    Like

    Josh

    May 9, 2011 at 4:06 pm

  53. “In Durkheim’s class, there was Henri Bergson…”

    Under the standards applied by many universities today, Bergson would have made the top ten percent of his class (and thus “world class university material”), and Durkheim would not. The exact point being made is that there is a lot of variance in this.

    Einstein, by the way, was fourth out of five in his undergraduate class. Name the first three without looking them up.

    Like

    Guillermo

    May 9, 2011 at 4:25 pm

  54. “Strong GRE’s might indicate….an ability to memorize data and methods?”

    In many cases, they indicate a student took time to reverse-engineer the test by buying one of those prep books.

    Like

    Guillermo

    May 9, 2011 at 4:26 pm

  55. Digression, sorry:

    “Durkheim founded French sociology”

    Actually, no. Gabriel Tarde was the dominant figure in French sociology when Durkheim appeared on the scene. Similarly, Weber went against the hegemony of Werner Sombart.

    Like

    Guillermo

    May 9, 2011 at 4:35 pm

  56. “Willingness to work hard and jump through the hoops necessary for one to pursue the career their passionate about is, I think, an incredibly important thing…”

    True, but if the hoops are more bureaucratic than actually useful for one’s preparation, they are going to divert a lot of valuable people away from the intended goal. The thing is, standardized scores are meant to measure aptitude (not preparation) and they do not even do this well, judging by the success of materials that tell you how to “crack” them.

    Like

    Guillermo

    May 9, 2011 at 4:41 pm

  57. A theory of how to make the best decision of who to admit out of an applicant pool is different than the factors that lead to a successful PhD. Isn’t faculty dedication and attention as important as student quality? What about program quality? The emphasis on the quality of the student seems quite liberal, as if what the student brings with them is the whole story. Given the lack of variation within the pool of advisees each faculty has, how can we assess the relative importance of student vs faculty quality?

    Like

    Residual

    May 11, 2011 at 12:48 am

  58. […] Fabio Rojas provoked an intense discussion about GRE scores – here and here.  Initially, I did not understand why a number of responders seemed to be disappointed […]

    Like

  59. As a prospective grad applicant at some of your fine institutions, I’m not sure what it will say about me that I spent time reading this post with my GRE coming up in 48 hours –

    One of the criteria someone suggests as a major factor in grad student success is “tolerance for ambiguity” – the GRE tests just about anything but – as an east-of-the-Atlantic applicant who’s never taken a multiple choice test in her life before, the idea of not being able to say “actually it could be a or it could be b, and here are my reasons…” really goes against my nature – what do you mean there’s only one answer? Have you not heard the enlightenment is over?

    Like

    niriain

    June 13, 2011 at 11:57 pm

  60. Soc. is not my field, but I am curious: What do you do when confronted with an applicant with a lower GPA but solid grades the last 2 years of undergrad? What about a non-traditional applicant who pursues a masters and applies to Ph.D. programs 4-6 years out of undergrad? Is GPA still considered the most predictive indicator of success in this case?

    Like

    Mealy

    August 9, 2011 at 1:25 pm

  61. Mealy, that’s a good question. I don’t know how or if these studies account for time spent in the labor market. You should read the studies and see what they say.

    Like

    fabiorojas

    August 9, 2011 at 3:44 pm

  62. This post couldn’t be more right.

    Like

  63. I could not think you are more right..

    Like

    Satire

    December 15, 2011 at 6:06 pm


Comments are closed.