Archive for the ‘research’ Category

new book spotlight: approaches to ethnography

New book alert!  For those prepping a methods course or wanting additional insight into ethnography as a research method, sociologists Colin Jerolmack and Shamus Khan*  have co-edited an anthology Approaches to Ethnography: Analysis and Representation in Participant Observation (2017, Oxford University Press).**


In Approaches to Ethnography, several ethnographers, including myself, have contributed chapters that delve into our experiences with ethnography across the subfields of urban sociology, poverty and inequality, race and ethnicity, culture, political economies, and organizational research.  For example, in his chapter, Douglas Harper explains how he integrated visual ethnography to get farmers to discuss experiences of farming past and present, capture the itinerant lives and transitory relations among tramps, and document food traditions in Bologna, Italy.

My own chapter “Capturing Organizations as Actors” was particularly difficult to write, with several major chunks jettisoned and sections rewritten several times to incorporate feedback from an ever-patient Khan.  Eventually, I realized I was struggling with how to advocate what is taken-for-granted among organizational researchers.  Normally, organizational researchers write for audiences who readily accept organizations as the unit of analysis and as important and consequential actors worthy of study.  However, for sociologists and social scientists who are not organizational researchers, the organization falls into the background as static, interchangeable scenery.  Given this anthology’s audience, I had to make an explicit argument for studying organizations to readers who might be inclined to ignore organizations.

With this in mind, my chapter focused on explaining how to use ethnography to bring organizations to the foreground.  To illustrate how researchers can approach different aspects of organizations, I drew on my ethnographic data collected on the Burning Man organization.  Most of the vignettes tap never-before-seen data, including discussions from organizers’ meetings and my participant-observations as a volunteer in Playa Info’s Found.  With these examples, I show how organizational ethnography can help us understand:

  • how informal relations animate organizations
  • how organizations channel activities through routines and trainings
  • how organizations and its subcultures communicate and inculcate practices
  • how organizations handle relations with other actors, including the state

Here is Approaches to Ethnography‘s table of contents:

Introduction: An Analytic Approach to Ethnography
Colin Jerolmack and Shamus Khan

1. Microsociology: Beneath the Surface
Jooyoung Lee
2. Capturing Organizations as Actors
Katherine Chen

3. Macro Analysis: Power in the Field
Leslie Salzinger and Teresa Gowan

4. People and Places
Douglas Harper

5. Mechanisms
Iddo Tavory and Stefan Timmermans

6. Embodiment: A Dispositional Approach to Racial and Cultural Analysis
Black Hawk Hancock

7. Situations
Monica McDermott

8. Reflexivity: Introspection, Positionality, and the Self as Research Instrument-Toward a Model of Abductive Reflexivity
Forrest Stuart

* Jerolmack and Khan have also co-authored a Socius article “The Analytic Lenses of Ethnography,” for those interested in an overview.

** I have a flyer for a slight discount that I hope is still good from the publisher; if you need it, send me an email!


Written by katherinechen

January 13, 2018 at 4:55 pm

teaching archival methods for graduate students

In an interesting twist, I am teaching a graduate course in qualitative methods. Because many of our ethnographers are on sabbatical, someone needed to offer qualitative methods. So I am offering a course on archival methods.

It’s very, very rare that a sociology program will offer a course on this topic. It is also fairly rare that library science programs will offer one because most librarians and archivists are trained in records management, not research applications. So I basically just had to develop the course from scratch.

  1. Textbook: I decided to treat this as a research method course. So I chose one book that was a nice overview of conceptual issues in social research  methods. I chose Thinking Through Methods, by John Levi-Martin. Informal, fun and packed with good thinking.
  2. Other readings: Each week we’ll read a chapter or two from Martin’s book but I also added other topics. For example, the newsletter of the ASA section on historical comparative research had a great symposium circa 2005 where people discussed access issues. Another week, we’ll do some basic readings about IRB and human subjects issues.
  3. Course topics: Aside from general discussions of research method, we’ll cover the following,
    • Traditional archival work – how to identify, access, search, and analyze paper documents.
    • Content analysis – a few lectures on taking qualitative materials and reliably coding them.
    • Computational methods – a lecture or two on the basic of how to upload textual materials in large quantities and analyze them.
  4. Assignments: As usual, there is class participation and weekly summaries of the readings. But we have three major assignments:
    • The instructor will assign you a book based on archival materials. Read it, summarize and discuss how well the archival materials were used.
    • The instructor will pick an online archive (The Martin Luther King, Jr. Archive) and you will develop and answer a sociological question using the archive.
    • The student will develop their own social science question and topic for a term paper. But they must answer it with archival research from a collection housed at the Indiana University archives.

We have ten students, most from sociology & education, a few from library science and two miscellaneous students. I think it will be very interesting.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome!

Written by fabiorojas

January 11, 2018 at 5:01 am

biology and gender differences in personality

Andy Perrin responded on Scatterplot to a Twitter debate that happened yesterday and I couldn’t resist adding two cents. It started with a link posted by Nicholas Christakis to a review article on gender differences in personality across societies. The main claim of the article is that gender differences in personality are larger in more gender-egalitarian countries, providing support for evolutionary theories of gender differences and against social role theories, which would predict gender convergence in egalitarian countries.

Steve Vaisey commented, “I am genuinely interested to hear how sociologists who study gender would react to these findings.” Andy’s response, which is as usual worth reading, argued against the study’s interpretation from a social constructionist perspective.

This sent me down the rabbit hole of actually reading the article and a bit of the research it is based on. I am not impressed.

Let me qualify that I am not a gender scholar, nor am I deeply familiar with this literature. My priors are that gender differences in personality are both biological and social, but that the average person (obviously not everyone) is insufficiently skeptical of biological explanations because they fit our expectations and stereotypes. I am inclined to be doubtful about this kind of research, but open to evidence.

Here’s two reasons this article left me underwhelmed.

First, the lit review is sloppy in a way that makes me not trust the authors about other things — say, the quality of their data collection across relatively small samples in dozens of countries. It sets up social role theory as a straw man (“social role theories of gender development contend any and all ostensible differences between men and women are primarily the result of perceived gender roles” [p. 47], when the debate is really about relative importance, not “any and all ostensible differences”). It quotes from an article about social role theory (“men and women have inherited the same evolved psychological dispositions” [p. 47]), but the page is not part of the article, and the quote does not seem to appear in the article at all. Based on Googling, it appears to come from a misquote in an edited volume. This may seem trivial, but if you’re asking me to trust that you used good research methods on a study that involved data collection in 50+ countries, I’d like to know that I can count on you to represent the literature accurately.

Second, I dug into one of the more prominent empirical studies in the review, by the same lead author. The review describes the study like this:

More egalitarian gender roles, gender socialization and sociopolitical gender equity, however, were associated with larger gender differences. For example, the largest overall gender differences in personality were found in relatively high gender egalitarian cultures of France (d = −0.44) and the Netherlands (d = −0.36), whereas the smallest gender differences were found in the relatively low gender egalitarian cultures of Botswana (d = 0.00) and India (d = −0.01).

The examples here are somewhat cherry-picked. Yes, these are the top two and bottom two countries. But if you look at their whole chart (p. 173), there’s more to the story. The five countries with the biggest gender differences in personality (measured as mean gender difference in big-five traits) are France, Netherlands, the Czech Republic, Brazil, and Belgium. The five most gender-similar countries are Indonesia, Congo, Fiji, Botswana, and Finland.

If we look at the World Economic Forum Global Gender Gap indicator, France is the only country on the top five that makes it into even the top 20 countries for gender equality. Brazil and the Czech Republic are both basically at the global median. Finland, which has among the most similar personality scores by gender, is the third most gender-equal country in the world. So while correlations may be there across all countries, this is hardly dispositive unless you can come up some explanation for why gender egalitarianism leads to personality difference across genders in France, but similarity in Finland.

And when they actually dig into the relationships, they appear to be flimsy and overstated. The study examines nine country-level measures of gender equality. After controlling for the country’s development level, only four measures show a significant relationship to the gender-personality gap at the p < 0.05 level (p. 177). So by most measures, the study finds no relationship. Notably, neither of the UN composite measures of gender equality, the Gender Empowerment Measure and Gender Development Index, are significantly related.

The four measures that appear to be related to mean country-level gender difference in personality are 1) traditional values, 2) cultural trust (“can most people be trusted”), 3) the gender gap in smoking, and 4) “when the respondents are more inclined to agree with a question irrespective its content” [sic — I don’t even know what that means]. Two of these seem very questionably related to gender, and I’m going to discount those.

That leaves us with the following: countries with more traditional values have less sex differentiation in personality traits, and countries where women smoke a lot have more sex differentiation in personality traits. Both relationships are significant at the p < 0.05 level — but note that we have already run through several possible measures that turned out not to be significant.

From this, the study winds back up to its dramatic conclusion: “in more prosperous and egalitarian societies the personality profiles of men and women become decidedly less similar” (p. 178). More prosperous, yes: development level, not gender equality, is the best explainer of gender-personality differences. But more egalitarian? Color me unconvinced.

And from that, the abstract jumps to: “It is proposed that heightened levels of sexual dimorphism result from personality traits of men and women being less constrained and more able to naturally diverge in developed nations.” Wow. The word “naturally” is doing an awful lot of work there.

The review article covers a bunch of other studies, too, but based on what I’ve read it does not seem worth the time to dive in further. My priors — that personality differences are both biological and social, and that people are too credulous of biological explanations of gender differences — remain unchanged.


Written by epopp

December 20, 2017 at 9:01 pm

why your asa section should open its paper award

I guess I’m blogging again. I went off on this on Twitter, so thought I might as well throw it up on here too.

At ASA next week, SocArXiv is meeting with nine different sections to talk about the possibility of “opening” section paper awards. What does this mean? We’d like to see ASA sections make posting papers on SocArXiv part of the award nomination process. So if you wanted your paper to be considered for an award, you’d put it on SocArXiv, tag it “OOWScottAward” (or whatever), and that’s it. The rest of the process works the same.

Why is this a good idea? We believe that academic research shouldn’t be paywalled, and that it shouldn’t take years for research to reach an audience. Right now, academia is locked into a publishing system that relies on the labor of academics, paid for by universities, government, and the individuals themselves to make large profits for private companies. It makes universities pay through the nose so academics can read their own work, and makes it even harder for people with no academic affiliation, or an underresourced library, to access. This is not good for sociology or for academia, and it’s just not necessary. Getting the work out there, where colleagues and a broader audience can access it, isn’t that hard.

Many sociologists support greater openness. A fair number post their work on their own websites, or at, or elsewhere. But there is real value in having the work all in one place, and having that be a place that is committed to open science, rather than to monetizing your account.

By linking section awards to open access, ASA sections can help nudge sociology in this direction. Uploading to SocArXiv isn’t hard to do, but there’s an inertia factor to overcome. And since people want to win section awards, section award submissions are a good moment for overcoming it. If your paper is worth considering for an award, it should be worth sharing, and sections can help make this happen.

Making award-nominated papers open isn’t only good for the discipline, though. It’s good for the section, too. Having served on way too many section award committees in the last decade, I know that reading nominated papers is a great way to keep up with what’s going on in a subfield. This is often even more true of grad student submissions, which show you where the field is going. Why not get this great work out there sooner, and let people know the exciting things that are going on in your part of the discipline?

To sweeten the pot, SocArXiv is putting up $400 toward conference travel for the award winner of any “open” section award. We will also provide $250 of support for any individual award winner who uploaded their paper at the time they submitted to a nonparticipating section.

So if you think advancing openness is a good thing, and want to see your ASA sections support it, let them know. And if you have hesitations, bring them up in the comments — some we may be able to address, and we’d like to learn more about concerns we may not have anticipated.

(Curious what’s on SocArXiv? Here’s a few orgtheory relevant papers posted this summer:

Want more details about what SocArXiv is? Click here. Or how award opening works? See this blog post. Or ask in the comments.)


Written by epopp

August 6, 2017 at 12:28 pm

why do some states have higher mass incarceration than others? comment on pamela oliver’s blog

Over at Race, Politics, and Justice, Pamela Oliver asks why her home state of Wisconsin has such high rates of Black imprisonment in comparison to other states, even in times when rates are falling:

Wisconsin has stayed at the top of the pile in Black incarceration even though its Black incarceration rate has been declining. How can this be? The answer is that all the other states have been declining faster. By putting a scatter plot of state imprisonment rates on consistent axes, I’ve been able to produce a really cool animation effect.  The data source is the\ National Corrections Reporting Program public “in custody” file. Rates are calculation on entire population (all ages). States voluntarily participate in this data collection program and appear and disappear from the plot depending on whether they reported for the appropriate year. States are also eliminated if more than 10% of their inmates are recorded as having unknown race. You’ll see if watch long enough that the relative positions of most states stay the same, but the whole distribution starts  moving downward (lower Black incarceration rates) and to the left (lower White incarceration rates) in the last few years. You may download both these images and explanatory material in PDF format  using this link.

Interesting. This is a classic example of the “dog that didn’t bark.” What happened in other states that did not happen in Wisconsin? A few hypotheses: Wisconsin reflects particularly bad conditions in segregated places like Milwaukee; fixed effects of prosecutors – Wisconsin district attorney’s are notoriously bad; police enforcement is unusually harsh. Add your hypotheses or explanation in the comments.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street 


Written by fabiorojas

October 6, 2016 at 12:29 am

bad reporting on bad science

This Guardian piece about bad incentives in science was getting a lot of Twitter mileage yesterday. “Cut-throat academia leads to natural selection of bad science,” the headline screams.

The article is reporting on a new paper by Paul Smaldino and Richard McElreath, and features quotes from the authors like, “As long as the incentives are in place that reward publishing novel, surprising results, often and in high-visibility journals above other, more nuanced aspects of science, shoddy practices that maximise one’s ability to do so will run rampant.”

Well. Can’t disagree with that.

But when I clicked through to read the journal article, the case didn’t seem nearly so strong. The article has two parts. The first is a review of review pieces published between 1962 and 2013 that examined the levels of statistical power reported in studies in a variety of academic fields. The second is a formal model of an evolutionary process through which incentives for publication quantity will drive the spread of low-quality methods (such as underpowered studies) that increase both productivity as well as the likelihood of false positives.

The formal model is kind of interesting, but just shows that the dynamics are plausible — something I (and everyone else in academia) was already pretty much convinced of. The headlines are really based on the first part of the paper, which purports to show that statistical power in the social and behavioral sciences hasn’t increased over the last fifty-plus years, despite repeated calls for it to do so.

Well, that part of the paper basically looks at all the papers that reviewed levels of statistical power in studies in a particular field, focusing especially on papers that reported small effect sizes. (The logic is that such small effects are not only most common in these fields, but also more likely to be false positives resulting from inadequate power.) There were 44 such reviews. The key point is that average reported statistical power has stayed stubbornly flat. The conclusion the authors draw is that bad methods are crowding out good ones, even though we know better, through some combination of poor incentives and selection that rewards researcher ignorance.



The problem is that the evidence presented in the paper is hardly strong support for this claim. This is not a random sample of papers in these fields, or anything like it. Nor is there other evidence to show that the reviewed papers are representative of papers in their fields more generally.

More damningly, though, the fields that are reviewed change rather dramatically over time. Nine of the first eleven studies (those before 1975) review papers from education or communications. The last eleven (those after 1995) include four from aviation, two from neuroscience, and one each from health psychology, software engineering, behavioral ecology, international business, and social and personality psychology. Why would we think that underpowering in the latter fields at all reflects what’s going on in the former fields in the last two decades? Maybe they’ve remained underpowered, maybe they haven’t. But statistical cultures across disciplines are wildly different. You just can’t generalize like that.

The news article goes on to paraphrase one of the authors as saying that “[s]ociology, economics, climate science and ecology” (in addition to psychology and biomedical science) are “other areas likely to be vulnerable to the propagation of bad practice.” But while these fields are singled out as particularly bad news, not one of the reviews covers the latter three fields (perhaps that’s why the phrasing is “other areas likely”?). And sociology, which had a single review in 1974, looks, ironically, surprisingly good — it’s that positive outlier in the graph above at 0.55. Guess that’s one benefit of using lots of secondary data and few experiments.

The killer is, I think the authors are pointing to a real and important problem here. I absolutely buy that the incentives are there to publish more — and equally important, cheaply — and that this undermines the quality of academic work. And I think that reviewing the reviews of statistical power, as this paper does, is worth doing, even if the fields being reviewed aren’t consistent over time. It’s also hard to untangle whether the authors actually said things that oversold the research or if the Guardian just reported it that way.

But at least in the way it’s covered here, this looks like a model of bad scientific practice, all right. Just not the kind of model that was intended.

[Edited: Smaldino points on Twitter to another paper that offers additional support for the claim that power hasn’t increased in psychology and cognitive neuroscience, at least.]


Written by epopp

September 22, 2016 at 12:28 pm

Appetite for Innovation: Creativity & Change at elBulli (To be published by Columbia University Press on July 12, 2016)

How is it possible for an organization to systematically enact changes in the larger system of which it is part? Using Ferran Adria’s iconic restaurant “elBulli” as an example of organizational creativity and radical innovation, Appetite for Innovation examines how Adria’s organization was able to systematically produce breakthroughs of knowledge within its field and, ultimately, to stabilize a new genre or paradigm in cuisine – the often called “experimental,” “molecular,” or “techno-emotional” culinary movement.

Recognized as the most influential restaurant in the world, elBulli has been at the forefront of the revolution that has inspired the gastronomic avant-garde worldwide. With a voracious appetite for innovation, year after year, Adrià and his team have broken through with new ingredients, combinations, culinary concepts and techniques that have transformed our way of understanding food and the development of creativity in haute cuisine.

Appetite for Innovation is an organizational study of the system of innovation behind Adrià’s successful organization. It reveals key mechanisms that explain the organization’s ability to continuously devise, implement and legitimate innovative ideas within its field and beyond. Based on exclusive access to meetings, observations, and interviews with renowned professionals of the contemporary gastronomic field, the book reveals how a culture for change was developed within the organization; how new communities were attracted to the organization’s work and helped to perpetuate its practice, and how the organization and its leader’s charisma and reputation were built and maintained over time. The book draws on examples from other fields, including art, science, music, theatre and literature to explore the research’s potential to inform practices of innovation and creativity in multiple kinds of organizations and industries.

The research for Appetite for Innovation was conducted when Adria’s organization was undergoing its most profound transformation, from a restaurant to a research center for innovation, “elBulli foundation”.  The book, therefore, takes advantage of this unique moment in time to retrace the story of a restaurant that became a legend and to explore underlying factors that led to its reinvention in 2011 into a seemingly unparalleled organizational model.

Appetite for Innovation is primarily intended to reach and be used by academic and professionals from the fields of innovation and organizations studies. It is also directed towards a non-specialist readership interested in the topics of innovation and creativity in general. In order to engage a wider audience and show the fascinating world of chefs and the inner-workings of high-end restaurants, the book is filled with photographs of dishes, creative processes and team’s dynamics within haute cuisine kitchens and culinary labs. It also includes numerous diagrams and graphs that illustrate the practices enacted by the elBulli organization to sustain innovation, and the networks of relationships that it developed over time. Each chapter opens with an iconic recipe created by elBulli as a way of illustrating the book’s central arguments and key turning points that enable the organization to gain a strategic position within its field and become successful.

To find a detailed description of the book please go to:

Also, included Appetite for Innovation in its list of 17 books recommended for “creative leaders” to read this summer:





Written by M. Pilar Opazo

June 8, 2016 at 4:46 pm