the one where fabio stands up at the end of jess calarco’s job talk and yells, “j’accuse!!”
I was looking for trouble. I’d been drinking ginger ale all day and reading Andy Gelman blog posts. Then, the department email said some hot shot job candidate was giving a talk.
Jess Calarco strolls in Philly stlye and gives her job talk. A forty five minute talk on, of all things, ethnography. Give me a break! Little kids raising their hands, exercising their fancy-schmancy cultural capital. Don’t believe me? Go read it yourself – it’s in a new Oxford University Press book, called Negotiating Opportunity: How the Middle Class Secures Advantages in School. All the gory messy detail in 272 gripping pages of field work. Don’t buy the hardback for $99. Total rip-off. Get the more affordable paper back edition for $24.95!! Those publishers are total con-artists. You gotta be careful.
For the entire talk, Calarco goes on and on about how children from wealthier families negotiate the classroom in small incremental ways through student-teacher interaction. Asking for time on tests, arguing about assignments. What happens at the end of the talk has now become legend at IU soc. This is how Calarco remembers it:
Here is how I remember it. I straightened out my bow tie, I stood up, and asked: “The motivation for your field work is to understand how class based difference in class room interactional style might be linked to learning outcome or status attainment. What evidence do you have from field work that the association is present or explains the variation in outcome, much in the same way a quantitative researcher might use an R-squared to measure a model’s goodness of fit?”
You could hear a pin drop. Children cried. Snowflakes started melting. Then, after taking a few notes, Calarco calmly explains that she was collecting data on the student’s performance to examine the link between classroom behavior and achievement and then she summarized some initial thoughts.
FOILED AGAIN! My plan to undermine the discipline of sociology failed! I went back to my office and vented my frustration on anonymous job rumor websites.
+++++++
BUY THESE BOOKS!!
50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)
A theory book you can understand!!! Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)
The rise of Black Studies: From Black Power to Black Studies
Did Obama tank the antiwar movement? Party in the Street
Read Contexts Magazine– It’s Awesome!
orgtheory.net 
Philly style!
LikeLike
anon
May 17, 2018 at 4:33 am
Okay, I was kind of on team don’t-be-a-jerk before, but asking why someone thinks their causal claims are causal does not really seem to me like a “gotcha” question. Maybe it’s all in the delivery.
LikeLiked by 1 person
Elizabeth Popp Berman
May 17, 2018 at 1:40 pm
this is actually a good question. some folks might resort to claiming they are “just” interested in process. the question posed simply frames the issue of process in the context of to what extent that process captures the norm (versus the deviation). using quant words to “distract” a scholar is no different that using jargon to talk to someone in a different discipline or subfield. it is our duty to learn how to translate our work across various audiences. kudos to the candidate for not getting rattle. does not sound like she really answered the question, but this would not have even been a post if she had completely fallen apart or gotten defensive (which is what most candidates do and which is a total no-no).
LikeLike
Methods Matter
May 17, 2018 at 1:55 pm
I guess the potentially jerky part is that you don’t really need to frame the question in terms of R-squared in order to ask it, and so what you are testing is not only if the candidate has thought about evidence of causality (hopefully yes), but whether you can throw them by making them translate into a different methodological logic on the fly.
If you are “just” interested in process, though, I think you answer the question by explaining why you are prioritizing process over causality in this case, and maybe gesturing toward how you *would* look at causality, were that what you were trying to do.
LikeLiked by 1 person
Elizabeth Popp Berman
May 17, 2018 at 2:11 pm
Yes, it may be that Fabio’s mistake was in how he framed this whole discussion in his first post. Asking an ethnographer to think about the implied causality motivating their research is not “aggressive” nor “obnoxious” nor out of bounds. So don’t be a jerk is still right, but maybe Fabio’s not actually as much of a jerk as he presented himself to be.
OTOH, Colarco’s a rockstar, so not surprised she could handle it. The issue is about whether or not the person who couldn’t handle that phrasing is, indeed, a worse job candidate. Fabio–where’s your evidence of your model’s goodness of fit? :)
LikeLiked by 1 person
rorykramer
May 17, 2018 at 2:22 pm
If you didn’t want people to think your questioning was obnoxious or aggressive, but simply fair questions about people’s research, I don’t understand why in your initial blog post, you wrote the exact words “This strikes people as aggressive or obnoxious and they are right.” It feels like you wanted to provoke criticism, which you could then score easy points off of by showing examples of when you weren’t obnoxious or aggressive. But the only reason we all assumed you were being obnoxious or aggressive is because you told us you were!
Saying “I’m a jerk” and then responding when people criticize you for being a jerk by saying “I’m not a jerk” – it’s not a good look, in my opinion.
LikeLiked by 2 people
Justin
May 17, 2018 at 2:33 pm
The question has multiple motivations:
– Casual claims are mentioned or implied. What is the evidence for them, even just at the level of associations?
-Does this person in particular actually understand a range of research design issues?
– Is this the type of person who can have a constructive conversation with someone who is academically different than them? E.g., I am not a full time ethnographer*, can she actually have an intelligent discussion with me?
– If I move out of the “frame” (qual research), can they respond in a professional manner?
– One does not merely walk into a job talk and casually claim that brief student-teacher interactions lead to life long life status difference.
Delivery
– Like I said, I was wearing a bow tie and I went “Al Pacino” in the Q&A.
– IU is currently populated by some of the nicest and kindest people you will meet. Thus, it takes a while for people to move from “this is fantastic, tell me more” to “hold on, isn’t just a collinearity problem?” A more pointed question cuts (slightly) against the norm, but it helps us get into a more thorough investigation of the research a little bit quicker.
Philosophy:
– If someone says they’re a jerk, but it turns out they are not a jerk, are they somehow still a jerk?
* Though I have actually once published an ethnographic work and my work often includes some element of field work.
LikeLiked by 2 people
fabiorojas
May 17, 2018 at 3:36 pm
Let them drink fresh water.
LikeLiked by 1 person
ggauchat
May 17, 2018 at 3:41 pm
I’d be interested to know how often during job talks a scholar using quantitative methods is asked how an ethnographer would think about their problem, question, or results. My guess would be never, which speaks to the fact that quantitative methods are dominant in the discipline. I think the problem here is that quantitative and qualitative methods, arguments, and guiding assumptions are not commensurate in all cases and we can’t assume that someone trained primarily in one methodology understands these dimensions of the other. Quantitative methods come out of the positivist tradition whereas qualitative methods are interpretive and more recently post-positivist. In this case, rather than framing the question in terms of how a regression would look coming out of this ethnography, it might be more productive to ask what hypotheses the author could identify from the study’s conclusions that could then be tested using a quantitative design, or whether others have measured the kinds of processes the author analyzes. It might move the discussion toward how these methods can be integrated or more effectively inform one another. People like Mario Small and Sam Lucas have written about how to handle this practically and Matthew Desmond’s “Evicted” is a great example of how the two methods inform and can serve to verify one another. This to me seems like a more productive line of questioning and more fruitful for our collective enterprise than method A vs. method B.
LikeLiked by 3 people
Joel Stillerman
May 18, 2018 at 3:51 pm
Well presumably we all know that SES is correlated with life outcomes. What she was doing as an ethnographer was *suggesting* one possible mechanism (of many) that might contribute to an explanation of that correlation. Of course she couldn’t, doing ethnography, provide strong evidence for causation. But, short of natural experiments (eg. the Oregon medicaid study) it’s almost impossible in social science to strongly motivate causal mechanisms.
What social scientists do is find correlations, then try to come up with prima facie *plausible* mechanisms that seem to explain them. Lack of controlled experiments means not much happens beyond that, both in qualitative and in quantitative stuff.
LikeLiked by 1 person
Rodger
May 23, 2018 at 4:05 am