“Talk with your family about [Medicare] Part D over Thanksgiving dinner”: How markets require bounded relationality

 

Question: What do the following three scenarios have in common?

Scenario A.  Congrats, you’re turning 65 years old!

You’re turning 65 years old.  In the US, if you have worked enough units, you are eligible for Medicare; you must select health insurance by choosing among traditional Medicare and HMO plans.  You also need to choose insurance that will cover  your current or anticipated prescription medications.  Depending on where you live, this could involve comparing around 50 different plans.

You start by consulting the Medicare booklet and wading through the flood of mail from insurance providers.  Despite this information, you’re having difficulties understanding the differences among plans and determining how much plans will charge for your medications.  Moreover, you’re not quite sure which medications that you’ll need in the upcoming year.  Each year after this, you’ll have an almost two-month-long window for making these decisions – a period that is happening now, ending Dec. 7.

If you have a long life, you’ll have plenty of practice working with this market.  How do you select a plan appropriate to your needs right now and then in the future?

Scenario B.  Congrats, you’re getting ready to enter high school!

You are a student at a NYC public middle school.  Since students are not automatically assigned to public high schools, you and your family must choose from among 750 programs and rank order your choices.  (If you are two years old or older, your parents must do the same for public pre-K and kindergarten school programs.)  To learn about your options, you can look at a directory of descriptions of these programs and then research each school online.  If possible, you and your family will also attend information fairs and schools’ open houses and tours, where you might be asked to fill out additional forms or leave your information.

Some schools have different criteria for what kinds of prospective students they prioritize, and most selective programs don’t provide rubrics for how they rank prospective students – information crucial for ascertaining your chances of acceptance.  After you submit up to 12 rank-ordered choices, an algorithm, modelled after a medical residency matching program designed by a economist, will generate a match based on schools’ priorities and your listed options.  And, btw, charter schools and private schools have their own admissions processes and admissions deadlines.

How do you choose and rank public high school programs?  Should you try to maximize your choices by also applying to charter schools and, if you have the financial resources, private schools?

Scenario C.  Congrats, you’re rich!

You have amassed enviable, immense wealth.   But, your mattress is bursting, and you distrust regular banking.  And, for whatever reason, you’re not fond of having the state taking a portion to support the common good, social insurance, military spending, etc.  Thinking ahead, you worry about your family having unfettered access to your financial legacy; relatives might fritter away that wealth!  Also, you have a few relationships that other family members don’t (yet) know about, and you want to make sure that those loved ones are also taken care of after your inevitable passing.  So, what to do?

Answer: Most likely, you’ll need what I call “bounded relationality” to assist you with entering complex markets and making exchanges.   To explain what bounded relationality is, I’ll preview excepts from my advance, online first article “Bounded relationality: how intermediary organizations encourage consumer exchanges with routinized relational work in a social insurance market.”

The bounded relationality concept combines two of my favorite theories: (1) economic sociology’s relational work by Viviana Zelizer, Fred Wherry, and Nina Bandjel* and (2) Herbert Simon’s theory about how organizations compensate for people’s bounded rationality, or difficulties with making decisions.

During several years of my research on organizations that support older adults, I observed workshops and meetings for organizational representatives and professionals, including social workers, on topics such as how to select Medicare insurance plans.

At one of these workshops, a representative from the Centers for Medicare & Medicaid Services, described officials’ hopes that families would discuss prescription plans at family get-togethers: ‘We tried to say, “Talk with your family about [Medicare] Part D over Thanksgiving dinner,” but we don’t know if people did.’   His comment revealed how much the market relies upon relational work, or connections formed and sustained with other persons (Zelizer 2012) and organizations.

Using observations of US governmental, advocacy, and human service organizations’ (GAHSOs) talks, I show how these intermediary organizations endorsed “bounded relationality” when teaching conventions for participating in the market of social insurance.  Unlike conventional consumer goods and services markets, insurance options are difficult to evaluate and exchanges are challenging to switch.  Decisions are also consequential, with suboptimal decisions impacting personal well-being and requiring support or intervention by family members, if they are available.

Read more about bounded relationality after the jump:

“Organizations, Markets, & the State” course at the Graduate Center, CUNY, offered for this spring 2020

Are you a graduate student in the Inter-University Doctoral Consortium?*  If so, please consider taking my “Organizations, Markets, & the State” class at the Graduate Center, CUNY.   At student request, I am teaching this class on the sociology of organizations this spring 2020 on Wednesdays @11:45am-2:45pm. Our first class meets on Wed., Jan. 29, 2020.

 

In addition to covering the classics of organizational research, the course includes cutting edge organizational research.  The course also incorporates topics and assigned readings based on enrolled students’ interests.  When I’ve taught this class in past semesters, guest speakers, including Nicole P. Marwell, Jeff Sallaz, Michel Anteby, Caroline W. Lee, Frederick Wherry, Pilar Opazo, and Elizabeth Popp Berman, have discussed their research projects.  (And, Fabio Rojas joined us for a special get-together during a visit to NYC!)

One of the aims of the class, besides learning substantive content, is to develop a local community of emerging scholars whose relationships spanning local, US, and international boundaries.  So, if you are an organizations researcher who is located or will happen to be in the NYC area during spring 2020, please email me about presenting your research!  We’ve also learned about professional development with guests, as participants are eager to learn about different kinds of institutions and career paths.

 

Here is the spring 2020 course description:

Organizations, Markets, & the State, Spring 2020

Graduate Center

Prof. Katherine K. Chen

Course Description

How can people coordinate action across growing groups in creative versus conventional ways?

How can people organize in ways that widen versus reduce power differentials among members?

How do people and organizations hoard advantages for a select few versus ensuring more equal access to all?

How do organizations fend off versus embrace market ideology, and how do organizations encourage members to adopt these perspectives?

Organizations are crucial actors in contemporary society, and they are also sites where many of us expend significant efforts connecting with or coordinating collective action.  Despite their central role in shaping our experiences from education to the workplace to governance, organizations are often overlooked or taken-for-granted among researchers and laypersons.  When researchers do study organizations, they typically pay little critical attention to power dynamics and organizing possibilities.

Building upon more critical perspectives, participants will learn why organizations form, how they develop, and how they can exacerbate or alleviate inequalities.  We will also discuss organizations’ relations with the state and markets, and how these relations affect action.  We will cover a variety of organizational forms, from conventional bureaucracies to networked firms to democratic organizations, with a focus on participants’ organizational fields of interest.  Theories studied incorporate the classics, as well as cutting edge synthetic work like Strategic Action Fields (SAFs), racialized organizations, and relational inequality theory (RIT)’s inequality-generating mechanisms.   Methodological approaches covered include ethnography, interviews, and other qualitative methods, and quantitative analyses.

This course supports deepening participants’ substantive knowledge, including preparing for comprehensives, extending cross-over expertise in a substantive area (i.e. social movements, urban sociology, stratification, education, cultural sociology, etc.), and designing and carrying out research.  In addition, this course aims to both promote professional development and forming a community of supportive scholars for emerging research.

*If you are a student at one of the below schools, you may be eligible, after filing  paperwork by the GC and your institution’s deadlines, to take classes within the Consortium:

Columbia University, GSAS
Princeton University – The Graduate School
CUNY Graduate Center
Rutgers University
Fordham University, GSAS
Stony Brook University
Graduate Faculty, New School University
Teachers College, Columbia University
New York University, GSAS, Steinhardt

“don’t be afraid to push big, bold projects” and “be brave and patient”: Dustin Avent-Holt and Donald Tomaskovic-Devey on producing Relational Inequality Theory (RIT)

 

Dustin Avent-Holt and Donald Tomaskovic-Devey, who collaboratively published their book Relational Inequalities: An Organizational Approach (Oxford University Press), graciously agreed to do a joint email interview with orgtheory!  Here, we discuss their book and the process leading up to the production of the book.  Readers who are thinking of how to apply relational inequality theory (RIT), join and bridge scholarly conversations, and/or handle collaborative projects, please take note.

First, I asked Dustin and Don substantive questions about RIT.  Here, both authors describe how they used their workplaces in higher education as laboratories for refining their theory.  Also, Don channeled his disappointment with the limits of Chuck Tilly’s Durable Inequalities into fueling this endeavor.

1. Katherine.  How did you apply the insight of relational inequality in your own lives?  For example, both of you are at public universities – how does knowing relational inequality affect your ways of interacting with other people and institutions?

Dustin. I think for me one of the ways I see this is becoming faculty during the process of writing the book and being in a transitioning institution. I was hired out of grad school to Augusta University when it had just merged with the Medical College of Georgia. With this merger, Augusta University moved from being a teaching-focused college to a comprehensive research university that includes both graduate and undergraduate programs and a mission focused on research. Experiencing this transition  made me think through the daily lives of organizations in a much less structural way as I saw people negotiating and renegotiating the meaning of the institution, the practices and policies, creating new ways of fulfilling institutional roles, etc. I guess in that way it highlighted the work of Tim Hallet on inhabited institutionalism. As university faculty and staff, we didn’t just copy a bunch of templates from the environment, people were translating them and challenging them in the organization. And we still are, 7 years later, and I suspect we will be for a very long time. Organizations at that moment became enactments rather than structures for me, something to be relationally negotiated not simply imported. Don and my endeavor then to understand inequality in this context actually began to make more sense. And in fact during our weekly conversations about the book, I do remember often relating stories to Don of what was going on, and this certainly shaped how I thought about the processes we were thinking through.

I don’t know if that is what you were after in your question, but it is for me this experience shaped how I have come to think about organizations, and became central to how we think about organizations in the book. 

Don. No fair, actually apply a theory in our own lives? Seriously though, I became pretty frustrated with the black hole explanations of local inequalities as reflecting “structure” or “history”. These can be analytically useful, but simultaneously disempowering. Yes, some students come to the University with cultural capital that matches some professors, but this does not make them better students, just relationally advantaged in those types of student-teacher interactions. At the same time the University exploits revenue athletes for its purposes while excluding many others from full participation. The struggles of first gen students and faculty are produced by relational inequalities. 

As a department chair I was keenly aware of the university dance of claims making around status and revenue and that this had to be actively negotiated if our department was going to be able to claim and sequester resources. This sounds and to some extent is harsh, since success might mean taking resources indirectly from weaker or less strategic departments, although it can also feel insurgent if the resource appears to be granted or extracted from the Provost. But the truth is that university resources flow in a complex network of relationships among units, students, legislators and vendors (beware the new administrative software contract!). 

The Dean will pretend this is about your unit’s “productivity”, it’s never that simple.*  It’s also great to have allies, at UMass we have a great faculty union that works to level the playing field between departments and disrupt the administrative inequality dance.

* Katherine’s addition: Check out this satirical twitter feed about higher ed administration for laugh/cries.

hot off the press: NVSQ special issue on “nonprofits and policy”

As Aug. ends, now’s the time to squeeze in that last bit of reading and consider new additions to course syllabi before the new semester’s start.

The Association of Research on Nonprofit Organizations and Voluntary Associations (ARNOVA)‘s flagship journal Nonprofit and Voluntary Sector Quarterly has just published a special issue on “Nonprofits and Policy.” This issue was sponsored by the Kresge Foundation and guest edited by political scientist Steven Rathgeb Smith and sociologist/nonprofit organizations researcher Kirsten A. Grønbjerg.

This special issue’s articles include:

• Celina Su’s cutting edge research on New York City’s experiment with participatory budgeting,
• Kimberly Wiley and Frances Berry’s study of how nonprofit domestic violence organizations handle competing demands of funders (with a surprising finding, at least for those readers who are skeptical of organizations’ capacities to serve marginalized populations),
• My ethnographic research applying Fligstein and McAdam’s Strategic Actions Fields (SAFs) theory to explain how organizations that help people age in place reacted to policy changes.

socarxiv highlights, asa edition

In honor of ASA, which starts on Saturday, I’m highlighting a handful of SocArXiv papers that will be presented at the conference. Their time/location is noted below as well. If you’ve just shared an ASA paper of your own with your discussant (and if you haven’t, time to get moving), consider uploading it to SocArXiv as well. You can always update it with a revised version later.

A related note—as I’ve been collating these the past few months, I’ve been noticing a pretty heavy gender imbalance in my selections, even though I’ve been paying attention. At first I thought it was my subfield tastes or implicit bias, but looking more closely, the pool itself is quite male-dominated—certainly more so than the discipline as a whole. So women in particular, please consider sharing your papers!

And last point—a few days ago I noticed that a version of a SocArXiv paper by Penn State demographer Alexis Santos-Lozada and colleague Jeffrey T. Howard on excess deaths in Puerto Rico after Hurricane Maria, was just published in JAMA. Santos-Lozada’s research was highlighted here last month pre-publication. Congratulations—it’s important work.

Standard disclaimer: I make no claim to peer review or formal evaluation of the papers here. Read it yourself before you cite.

Market crises as disasters: the social meaning of financial risk in 401(k) retirement accounts

Adam Hayes

Section on Economic Sociology Refereed Roundtable Session

Mon, August 13, 4:30 to 5:30pm, Pennsylvania Convention Center, Level 100, 113A

This very interesting paper links econ soc with the sociological literature on disasters to understand how experience of a stock market crash causes households to shift their 401(k) investments toward long-term conservatism. This is consistent with neither neoclassical or behavioral economic predictions, but fits predictions regarding the social amplification of risk. Unfortunately, this social reaction may not bode well for retirement savers in the long run—not a good sign as bankruptcy rises among older Americans.

God’s Country in Black and Blue: How Christian Nationalism Shapes Americans’ Views about Police (Mis)treatment of Blacks

Samuel Perry, Andrew Whitehead, and Joshua Davis

Religion, Politics, and Donald J. Trump

Sat, August 11, 10:30am to 12:10pm, Philadelphia Marriott Downtown, Level 4, Franklin Hall 7

ASA’s theme this year is “Feeling Race”, and this paper on Christian nationalism and attitudes about police mistreatment of blacks is certainly relevant. Drawing on a national probability sample, it shows a relationship between Christian nationalism, measured by agreement with statements like “The federal government should declare the United States a Christian nation,” and beliefs about differences in how U.S. police officers treat blacks and whites. There is a strong relationship between Christian nationalism and believing the police treat people of all races similarly, unsurprisingly, but with some unexpected twists: the relationship declines with increasing religious activity, and it holds for nonwhite Christian nationalists as well as white ones.

Duality in Diversity: Cultural Heterogeneity, Language, and Firm Performance

Matthew Corritore, Amir Goldberg, and Sameer Srivastava

Culture and Organizations

Sat, August 11, 4:30 to 6:10pm, Philadelphia Marriott Downtown, Level 4, Franklin Hall 4

Much of the conversation around diversity—certainly the legal conversation around affirmative action—is grounded in the idea that diversity is beneficial for everyone in an organization. In terms of organizational capacity, the working assumption is usually that of a tradeoff between better coordination (with homogeneity) or more creative problem-solving (with diversity). This paper shifts that conversation by examining intrapersonal diversity—having individuals with more internally heterogenous beliefs. Drawing on data from Glassdoor, the paper argues that interpersonal heterogeneity worsens organizational coordination, while intrapersonal heterogeneity facilitates creativity. An interesting angle with implications for debates over the effects of diversity from Google to higher ed.

Gender trouble beyond the LGB and T: Gender Image and Experiences of Marginalization on Campus

Kari J. Dockendorff and Claudia Geist

Section on Sociology of Education Refereed Roundtable Session

Sun, August 12, 2:30 to 3:30pm, Philadelphia Marriott Downtown, Level 5, Salon H

Finally, Dockendorff and Geist survey a pool of American undergraduates to better understand the experiences of trans and nonbinary students, particularly focusing on students who report others perceiving them as androgynous or of a different gender than they perceive themselves, finding higher levels of self-reported marginalization among those who identify “beyond the binary”. The paper makes innovations in gender measurement as well as exploring student experiences around gender marginalization in new ways.

Enjoy the remaining days of summer, whether you’re heading to ASA in Philly, the Academy meetings in Chicago, or just lounging by the beach.

 

socarxiv highlights for july

Once again it’s time for monthly highlights from SocArXiv, aka Stuff Beth Thought Was Interesting. This month we’ve got an eclectic set of papers with no unifying theme, other than being uploaded to SocArXiv in the last month. As usual, I remind you that SocArXiv papers are not necessarily peer-reviewed, so use judgment when you read.

In Puerto Rico, Excess Deaths Following Hurricane Georges Persisted for Three Months

Alexis R. Santos-Lozada

In the craziness of political life these days, disasters fall off the radar way too quickly. But we still don’t know how many people died in Puerto Rico as a result of the hurricane last fall. The official death toll is still 64. But a number of scholars have been working to provide better estimates of the real impact of the tragedy. Another widely reported estimate, based on house-to-house survey data, produced a figure of 4600. Now this paper by Santos-Lozada, comparing death records from last year to historical averages for the three-month period following the hurricane, suggests a number around 900. Debate over the true number will continue, but this is a great example of the kind of paper that needs to get out there quickly, rather than lingering hidden in peer review.

The Sources and Political Uses of Ambiguity in Statecraft

Katrina Quisumbing King

Well, this is right up my alley. A large and growing literature examines how states make populations “legible” through censuses, mapmaking, data collection and so on. But it’s also clear that states use ambiguity—in laws, definitions, policies—in productive ways. This paper uses the historical case of the U.S. colonization of the Philippines to show how the institutionalization of ambiguity can resolve imperial conflicts. After the U.S. took the Philippines in the Spanish-American War, debates about what their status should be—a colony? an eventual U.S. state?—were resolved by creating ambiguous categories: the territory would be “foreign in a domestic sense” (according to the Supreme Court), and their residents neither citizens nor aliens. The paper goes on to explore how this institutionalized ambiguity helped the U.S. resolve competing, and contradictory, demands that it remain true to the Constitution while presenting Filipinos, perceived as racially inferior, from accessing the rights of citizens.

Addendum: I just noticed this paper just won the grad student paper award in political sociology, and an honorable mention for the comparative-historical grad award. See, I have good taste!

Inequality Is a Problem of Inference: How People Solve the Social Puzzle of Unequal Outcomes

Jonathan J.B. Mijs

How do we understand inequality? This interesting theoretical paper argues that if people’s informal theories of inequality shape their political views, we need to take more seriously the task of understanding where those theories come from. The paper suggests we do that by conceptualizing inequality beliefs as inference problems – that ordinary people look for theories that explain their everyday experiences and observations of inequality. Personally, I wouldn’t discount the extent to which we learn our theories explicitly from those around us, as well as inferring them from experience, but this is still an intriguing way to conceptualize a challenging problem.

Career Paths and Prospects in Academic Data Science: Report of the Moore-Sloan Data Science Environments Survey

Stuart Geiger, Charlotte Mazel-Cabasse, Chihoko Cullens, Laura Norén, Brittany Fiore-Gartland, Diya Das, and Henry Brady

Ready to ditch sociology entirely? How about a career in the growing field of data science? This report on three major data science institutes—at Berkeley, NYU, and the University of Washington—explains what data science is, what academic data scientists do, and presents interesting interview data on the career paths of early-career data scientists. If you think you might be a data scientist, or would like to be, this report is definitely worth a read.

Okay, that’s it for this month’s SocArXiv update. If you’re in the northeastern U.S., hope you’re staying cool.

SocArXiv highlights for May, education organizations editions

Among the many interesting papers posted to SocArXiv this month, this I’m highlighting four that circle around a loose theme: education organizations. All four this month are not-yet-published drafts, including three working papers and a very interesting dissertation chapter. As always, these are papers that caught my eye but they are not peer-reviewed; read them yourself before citing.

The Performativity of Organizational Culture in a No Excuses Charter School

Jason Radford

Okay, I’ll admit I was grabbed by the title. Love the concept or hate it, we usually associate “performativity” with economics or finance, at least since Donald MacKenzie’s work. But of course the old Thomas theorem—“If men define situations as real, they are real in their consequences”—applies to our understanding of the world more generally. So why not the performativity of org theory? Many innovative or experimental organizations are grounded in theories, academic or otherwise, about what accounts for organizational success. In the case at hand, No Excuses charter schools were founded based on a theory that strong leaders could create organizational cultures that would produce educational success for disadvantaged students. This ethnographic case study of one No Excuses school explores how leadership attempted to implement their theory and reacted when the theory did not quite perform in the way it expected.

Engineering a Platform: Constructing Interfaces, Users, Organizational Roles, and the Division of Labor

Shreeharsh Kelkar

This intriguing paper, also ethnographic, reports on the development of edX, a MOOC platform started in 2012 as a nonprofit collaboration between Harvard and MIT. edX began as an educational organization—it saw itself as linking faculty and TAs with software developers and students—but eventually became a platform for other organizations to use, adopting the slogan “We do software so that you can do education.” In the process, edX had to create a new division of labor in which it did software development (internally framed as “boring”), while instructors at partner institutions would do the “interesting” work of content creation. Yet the supposedly neutral role of platform design still had huge pedagogical implications, even as edX came to distinguish what it did as an organization from “education”. The paper concludes by arguing that understanding platforms requires attending not just to “licenses, legal arrangements, and calculative agencies, but also to the shaping of organizational roles within the eco-system.”

The Ties that Corporatize: A Social Network Analysis of University Presidents as Vectors of Higher Education Corporatization

Tressie McMillan Cottom, Sally Hunnicutt, and Jennifer A. Johnson

This new working paper from Cottom and colleagues reports on a social network analysis of a dataset involving the career histories and education of presidents of Association of Public & Land-grant Universities (APLU) members, including a number of HBCUs, and a smaller sample of for-profit institutions. Presidents of predominantly white APLU institutions are tightly networked, while for-profit institutions and HBCUs are marginal to the network; the institutions with the highest degree centrality are Purdue, Michigan, Georgia, North Carolina, and Texas A&M—an observation that makes Purdue’s recent experiments with purchase of the for-profit Kaplan to run its online classes all the more interesting.

Engineering Credentials: Educational Entrepreneurship as Statecraft in the Cold-War United States

Alexander Kindel and Mitchell Stevens

Last but not least, this historical case study of Stanford’s postwar expansion of graduate science and engineering education engages the longstanding “credential society” literature by advocating for greater attention to the role of the state. While credentialing theory gives credit to educational entrepreneurs and the status projects of occupational groups in driving the expansion of credential-producing programs, this paper points to the fundamental role of the state in this process—through massive government patronage that schools like Stanford entrepreneurially tapped into to drive their own credentialing contributions.

This is self-evidently true for the fields that benefited from the federal government’s massive Cold War expansion of science funding, but struck another chord with me as well, as I recently revised a draft chapter on the origins of public policy schools. Policy schools, which only got started in the late 1960s, were themselves an entrepreneurial response on the part of universities to the massive demand for RAND-style analysis produced by federal rollout of the Planning-Programming-Budgeting System in 1965—making them another, very distinct case in which educational entrepreneurship could be better described as academic statecraft.

Okay, that’s it for this round. Maybe now that the semester’s done I’ll find a way to start blogging again, but in the meanwhile, keep those papers coming!

socarxiv highlights for april

Welcome back for a second round of monthly SocArXiv highlights. This is a way to call out a handful of the many papers that were posted in April, focusing mostly and sociology and reflecting my totally idiosyncratic tastes. Some are working papers or forthcoming articles; some are preprints of recently published work. All are freely available via OSF.

Disclaimer: I make no claim to peer review or to evaluation of the papers here. Read it yourself before you cite!

The Price of an Uncertain Promise: Fair Value Accounting and the Shaping of Bank Counterparty Risk Valuation Practices

Taylor Spears

This paper, which lies at the intersection of social studies of finance and institutionalism/field theory, is a fascinating look at how the adoption of fair value accounting by the Financial Accounting Standards Board affected the financial modeling practices used by banks. Consistent with MacKenzie (2011), the paper finds competing and conflicting valuation processes within and across organizations, and that the new standards tipped the balance in favor of a set of practices aligned with financial economics. The paper does a really nice job of showing how institutional and sociomaterial explanations can be complementary, and that both are needed to understand this kind of change.

Cultural Meanings and the Aggregation of Actions: The Case of Sex and Schooling in Malawi

Margaret Frye

This paper was published in ASR last year, but it went up on SocArXiv this month, so fair game. Maggie Frye does great and original work linking cultural accounts and demographic data. By moving between empirical evidence on sexual behavior and school-leaving, and student/teacher accounts of why sexual relationships cause girls to leave school, Frye produces a compelling account of how causal narratives — even inaccurate ones — influence actions in ways that have population-level effects.

Two from the sociology of science:

Men Set Their Own Cites High: Gender and Self-citation across Fields and over Time

Molly King, Carl Bergstrom, Shelley Correll, Jennifer Jacquet, Jevin West

and

The Matthew Effect in Science Funding

Thijs Bol, Mathijs de Vaan, and Arnout van de Rijt

The findings of these two papers may not be shocking, but both provide important new evidence of the effects they describe. The King et al. paper, published in Socius last year, shows that men cite their own work 70% more than women, and that these numbers have not changed over the last fifty years. The Bol et al. paper, published this year in PNAS, shows that early career researchers just above the funding threshold of a major European grant accumulate twice as much funding over the next eight years as those just below it. The practical takeaway, though, is that part of the gap happens because initially unfunded applicants subsequently apply for fewer grants, not only because successful applicants are more likely to be funded down the road. So women, cite your own work, and rejected grant applicants, keep on trying.

Can Cultural Consumption Increase Future Earnings? Exploring the Economic Returns to Cultural Capital

Aaron Reeves and Robert de Vries

Just yesterday a graduate student asked me if anyone had looked at whether Lauren Rivera’s finding about the cultural matching that goes on at elite firms applies to other occupational settings. I said I didn’t know of work that did (though tell me if I’m wrong!), and then I ran into this paper, forthcoming in the British Journal of Sociology. While it doesn’t look at matching per se, it does examine whether cultural consumption predicts future earnings, upward social mobility, and promotions. (Answer: yes.) This seems like an area that is ripe for interesting work and where relationships are likely to vary a great deal across industry, occupation, and location.

Okay, that’s it for this time. Keep on posting your working papers and preprints to SocArXiv and I’ll keep on sharing — at least as much as I can.

submitted a paper for an ASA section award? submit it to SocArXiv and be eligible for a SOAR award too

If you’ve submitted a paper to be considered for an American Sociological Association section award – including a graduate student award – consider submitting it to SocArXiv as well. Any paper that is uploaded to SocArXiv by April 30 and wins a 2018 ASA section award will, upon letting us know, receive a supplementary SOAR (Sociology Open Access Recognition) award of $250 in recognition of your achievement. Support open access, gain recognition, and win money all at the same time!

Here’s how it works: You upload your paper to SocArXiv by April 30. If it’s a published paper, check your author agreement or the Sherpa/ROMEO database to see what version, if any, you’re allowed to share. Once you find out you’ve won a section award, email socarxiv@gmail.com. SocArXiv will send you a check for $250, as well as publicizing your paper and officially conferring a SOAR award. That’s the whole deal.

Sharing your paper through SocArXiv is a win-win. It’s good for you, because you get the word out about your research. It’s good for social science, because more people have access to ungated information. And now, with SOAR prizes for award-winning papers, it can be good for your wallet, too. For more information and FAQs visit this link.

 

SocArXiv highlights for march

SocArXiv has been up and running for a year and a half now, and has accepted well over 2000 papers to date. Although you can follow the SocArXiv bot on Twitter to see what’s coming down the transom, and this page provides a running feed of the latest papers and abstracts, it’s a lot to follow – last month more than 200 papers were uploaded.

Toward the end of making this firehose of research a bit more manageable, I thought I’d start to do a little curating. The intent at this point is to do this once a month, though clearly it could be a weekly feature.

Highlighted below are a handful of intriguing papers posted to SocArXiv recently. Selection criteria are totally idiosyncratic – sociology-centric and based on what looks intriguing to me, with some eye toward broader appeal. If you’re interested in helping to curate on a monthly basis, perhaps with a focus on a particular subfield, email me at epberman@albany.edu.

Disclaimer: I make no claim to peer review here or even to close reading of the papers to evaluate quality; some have been published and gone through peer review while others are working papers.

So, with no further ado:

The Emotional Labor of Surveillance: Evidence from the Fast Fashion Retail Industry

Madison Van Oort

This ethnography- and interview-based paper looks at just-in-time scheduling, biometric scanners, and point-of-sale metrics as forms of worker surveillance at two major “fast fashion” retailers. It details the ways these technologies shape work practices and require new kinds of emotional labor—the “emotional labor of surveillance.” I saw Van Oort present research from this project at ASA last year and it was fascinating – there is lots of room to understand how new technology is yet again restructuring the workplace through new forms of discipline than in turn produce their own resistance.

Exposure to Opposing Views can Increase Political Polarization: Evidence from a Large-Scale Field Experiment on Social Media

Christopher Bail, Lisa Argyle, Taylor Brown, John Bumpus, Haohan Chen, M.B. Fallin Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, Alexander Volfovsky

This large-scale experiment got a lot of attention on (surprise) social media when it was posted a couple of weeks ago. Following a survey, authors randomly assigned Democratic and Republican Twitter users to follow a bot that would periodically tweet messages from the “other side”. After a month, they surveyed respondents again, finding that Republicans became substantially more conservative after following a liberal Twitter bot, and Democrats became slightly (but insignificantly) more liberal. Make of that what you will, but it’s interesting experimental evidence.

The Rise of the Randomistas: On the Experimental Turn in International Aid

Kevin Donovan

Speaking of experiments, this paper recently published online-first in Economy and Society looks at how randomized controlled trials became a newly dominant form of knowledge in international development. Promoted as a means of securing epistemic certainty, RCTs have reconfigured both development economics and international aid itself, yet still fail to achieve the closure hoped for by their proponents. This intriguing paper, part of a not-yet-published issue on evidence-based policy, builds on the work of scholars like Monika Krause and Gil Eyal to understand how networks of expertise are produced and maintained.

Leaving the Financial Nest: Connecting Young Adults’ Financial Independence to Financial Security

Megan Doherty Bea and Youngmin Yi

As someone with an interest in student loans and their effects, I found this paper on young adults and their reliance on family support intriguing. Clearly the ability of parents to continue to assist young adults is a difficult-to-measure but important mechanism for the reproduction of inequality. Using PSID data, this paper uses group-based trajectory analysis to identify four latent classes of young adults: consistently independent, quickly independent, gradually independent, and consistently supported. The consistently independent group, with lower average socioeconomic status, reports more financial worry and has a greater chance of being in poverty. This approach seems very promising for better understanding the mechanisms through which intergenerational advantage is transmitted and reproduced.

Too Many Papers? Slowed Canonical Progress in Large Fields of Science

Johan S. G. Chu and James Evans

Finally, and appropriate to the project of sorting through lots of papers, this short working paper uses a very large dataset (57 million papers and a billion citations) to look at how scientific fields develop as the number of papers in them grows large. Increasing size leads to “ossification” of the literature rather than increased citation of new papers, suggesting that new ideas may have trouble gaining hold as readers, overwhelmed by the literature, focus on canonical texts. This intriguing evidence could be interpreted in a number of different ways, and will doubtless generate debate over which story best fits the empirical citation patterns.

There’s lots of good stuff out there – I easily could have highlighted several times this number of papers! Again, if you’re interested in helping curate interesting work on SocArXiv, please let me know – with more people, and different tastes, we could conceivably do something a little more systematic here.

one more day till oow award deadlines

Last call — March 31st is the deadline to nominate your work for the ASA Organizations, Occupations and Work section awards: the Richard Scott Article Award, the Max Weber Book Award, and the James D. Thompson Graduate Student Paper Award.

And while you’re at it, submit your paper to SocArXiv as well, where it will automatically become eligible for the SOAR (Sociology Open Award Recognition) awards — any paper already submitted to SocArXiv that wins any ASA section award is eligible for a supplementary $250 cash prize. Support open science and win money too!

new book spotlight: approaches to ethnography

New book alert!  For those prepping a methods course or wanting additional insight into ethnography as a research method, sociologists Colin Jerolmack and Shamus Khan*  have co-edited an anthology Approaches to Ethnography: Analysis and Representation in Participant Observation (2017, Oxford University Press).**

In Approaches to Ethnography, several ethnographers, including myself, have contributed chapters that delve into our experiences with ethnography across the subfields of urban sociology, poverty and inequality, race and ethnicity, culture, political economies, and organizational research.  For example, in his chapter, Douglas Harper explains how he integrated visual ethnography to get farmers to discuss experiences of farming past and present, capture the itinerant lives and transitory relations among tramps, and document food traditions in Bologna, Italy.

My own chapter “Capturing Organizations as Actors” was particularly difficult to write, with several major chunks jettisoned and sections rewritten several times to incorporate feedback from an ever-patient Khan.  Eventually, I realized I was struggling with how to advocate what is taken-for-granted among organizational researchers.  Normally, organizational researchers write for audiences who readily accept organizations as the unit of analysis and as important and consequential actors worthy of study.  However, for sociologists and social scientists who are not organizational researchers, the organization falls into the background as static, interchangeable scenery.  Given this anthology’s audience, I had to make an explicit argument for studying organizations to readers who might be inclined to ignore organizations.

With this in mind, my chapter focused on explaining how to use ethnography to bring organizations to the foreground.  To illustrate how researchers can approach different aspects of organizations, I drew on my ethnographic data collected on the Burning Man organization.  Most of the vignettes tap never-before-seen data, including discussions from organizers’ meetings and my participant-observations as a volunteer in Playa Info’s Found.  With these examples, I show how organizational ethnography can help us understand:

• how informal relations animate organizations
• how organizations channel activities through routines and trainings
• how organizations and its subcultures communicate and inculcate practices
• how organizations handle relations with other actors, including the state

Here is Approaches to Ethnography‘s table of contents:

Introduction: An Analytic Approach to Ethnography
Colin Jerolmack and Shamus Khan

1. Microsociology: Beneath the Surface
Jooyoung Lee
2. Capturing Organizations as Actors
Katherine Chen

3. Macro Analysis: Power in the Field
Leslie Salzinger and Teresa Gowan

4. People and Places
Douglas Harper

5. Mechanisms
Iddo Tavory and Stefan Timmermans

6. Embodiment: A Dispositional Approach to Racial and Cultural Analysis
Black Hawk Hancock

7. Situations
Monica McDermott

8. Reflexivity: Introspection, Positionality, and the Self as Research Instrument-Toward a Model of Abductive Reflexivity
Forrest Stuart

* Jerolmack and Khan have also co-authored a Socius article “The Analytic Lenses of Ethnography,” for those interested in an overview.

** I have a flyer for a slight discount that I hope is still good from the publisher; if you need it, send me an email!

teaching archival methods for graduate students

In an interesting twist, I am teaching a graduate course in qualitative methods. Because many of our ethnographers are on sabbatical, someone needed to offer qualitative methods. So I am offering a course on archival methods.

It’s very, very rare that a sociology program will offer a course on this topic. It is also fairly rare that library science programs will offer one because most librarians and archivists are trained in records management, not research applications. So I basically just had to develop the course from scratch.

1. Textbook: I decided to treat this as a research method course. So I chose one book that was a nice overview of conceptual issues in social research  methods. I chose Thinking Through Methods, by John Levi-Martin. Informal, fun and packed with good thinking.
2. Other readings: Each week we’ll read a chapter or two from Martin’s book but I also added other topics. For example, the newsletter of the ASA section on historical comparative research had a great symposium circa 2005 where people discussed access issues. Another week, we’ll do some basic readings about IRB and human subjects issues.
3. Course topics: Aside from general discussions of research method, we’ll cover the following,
   • Traditional archival work – how to identify, access, search, and analyze paper documents.
   • Content analysis – a few lectures on taking qualitative materials and reliably coding them.
   • Computational methods – a lecture or two on the basic of how to upload textual materials in large quantities and analyze them.
4. Assignments: As usual, there is class participation and weekly summaries of the readings. But we have three major assignments:
   • The instructor will assign you a book based on archival materials. Read it, summarize and discuss how well the archival materials were used.
   • The instructor will pick an online archive (The Martin Luther King, Jr. Archive) and you will develop and answer a sociological question using the archive.
   • The student will develop their own social science question and topic for a term paper. But they must answer it with archival research from a collection housed at the Indiana University archives.

We have ten students, most from sociology & education, a few from library science and two miscellaneous students. I think it will be very interesting.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome!

biology and gender differences in personality

Andy Perrin responded on Scatterplot to a Twitter debate that happened yesterday and I couldn’t resist adding two cents. It started with a link posted by Nicholas Christakis to a review article on gender differences in personality across societies. The main claim of the article is that gender differences in personality are larger in more gender-egalitarian countries, providing support for evolutionary theories of gender differences and against social role theories, which would predict gender convergence in egalitarian countries.

Steve Vaisey commented, “I am genuinely interested to hear how sociologists who study gender would react to these findings.” Andy’s response, which is as usual worth reading, argued against the study’s interpretation from a social constructionist perspective.

This sent me down the rabbit hole of actually reading the article and a bit of the research it is based on. I am not impressed.

Let me qualify that I am not a gender scholar, nor am I deeply familiar with this literature. My priors are that gender differences in personality are both biological and social, but that the average person (obviously not everyone) is insufficiently skeptical of biological explanations because they fit our expectations and stereotypes. I am inclined to be doubtful about this kind of research, but open to evidence.

Here’s two reasons this article left me underwhelmed.

First, the lit review is sloppy in a way that makes me not trust the authors about other things — say, the quality of their data collection across relatively small samples in dozens of countries. It sets up social role theory as a straw man (“social role theories of gender development contend any and all ostensible differences between men and women are primarily the result of perceived gender roles” [p. 47], when the debate is really about relative importance, not “any and all ostensible differences”). It quotes from an article about social role theory (“men and women have inherited the same evolved psychological dispositions” [p. 47]), but the page is not part of the article, and the quote does not seem to appear in the article at all. Based on Googling, it appears to come from a misquote in an edited volume. This may seem trivial, but if you’re asking me to trust that you used good research methods on a study that involved data collection in 50+ countries, I’d like to know that I can count on you to represent the literature accurately.

Second, I dug into one of the more prominent empirical studies in the review, by the same lead author. The review describes the study like this:

More egalitarian gender roles, gender socialization and sociopolitical gender equity, however, were associated with larger gender differences. For example, the largest overall gender differences in personality were found in relatively high gender egalitarian cultures of France (d = −0.44) and the Netherlands (d = −0.36), whereas the smallest gender differences were found in the relatively low gender egalitarian cultures of Botswana (d = 0.00) and India (d = −0.01).

The examples here are somewhat cherry-picked. Yes, these are the top two and bottom two countries. But if you look at their whole chart (p. 173), there’s more to the story. The five countries with the biggest gender differences in personality (measured as mean gender difference in big-five traits) are France, Netherlands, the Czech Republic, Brazil, and Belgium. The five most gender-similar countries are Indonesia, Congo, Fiji, Botswana, and Finland.

If we look at the World Economic Forum Global Gender Gap indicator, France is the only country on the top five that makes it into even the top 20 countries for gender equality. Brazil and the Czech Republic are both basically at the global median. Finland, which has among the most similar personality scores by gender, is the third most gender-equal country in the world. So while correlations may be there across all countries, this is hardly dispositive unless you can come up some explanation for why gender egalitarianism leads to personality difference across genders in France, but similarity in Finland.

And when they actually dig into the relationships, they appear to be flimsy and overstated. The study examines nine country-level measures of gender equality. After controlling for the country’s development level, only four measures show a significant relationship to the gender-personality gap at the p < 0.05 level (p. 177). So by most measures, the study finds no relationship. Notably, neither of the UN composite measures of gender equality, the Gender Empowerment Measure and Gender Development Index, are significantly related.

The four measures that appear to be related to mean country-level gender difference in personality are 1) traditional values, 2) cultural trust (“can most people be trusted”), 3) the gender gap in smoking, and 4) “when the respondents are more inclined to agree with a question irrespective its content” [sic — I don’t even know what that means]. Two of these seem very questionably related to gender, and I’m going to discount those.

That leaves us with the following: countries with more traditional values have less sex differentiation in personality traits, and countries where women smoke a lot have more sex differentiation in personality traits. Both relationships are significant at the p < 0.05 level — but note that we have already run through several possible measures that turned out not to be significant.

From this, the study winds back up to its dramatic conclusion: “in more prosperous and egalitarian societies the personality profiles of men and women become decidedly less similar” (p. 178). More prosperous, yes: development level, not gender equality, is the best explainer of gender-personality differences. But more egalitarian? Color me unconvinced.

And from that, the abstract jumps to: “It is proposed that heightened levels of sexual dimorphism result from personality traits of men and women being less constrained and more able to naturally diverge in developed nations.” Wow. The word “naturally” is doing an awful lot of work there.

The review article covers a bunch of other studies, too, but based on what I’ve read it does not seem worth the time to dive in further. My priors — that personality differences are both biological and social, and that people are too credulous of biological explanations of gender differences — remain unchanged.

why your asa section should open its paper award

I guess I’m blogging again. I went off on this on Twitter, so thought I might as well throw it up on here too.

At ASA next week, SocArXiv is meeting with nine different sections to talk about the possibility of “opening” section paper awards. What does this mean? We’d like to see ASA sections make posting papers on SocArXiv part of the award nomination process. So if you wanted your paper to be considered for an award, you’d put it on SocArXiv, tag it “OOWScottAward” (or whatever), and that’s it. The rest of the process works the same.

Why is this a good idea? We believe that academic research shouldn’t be paywalled, and that it shouldn’t take years for research to reach an audience. Right now, academia is locked into a publishing system that relies on the labor of academics, paid for by universities, government, and the individuals themselves to make large profits for private companies. It makes universities pay through the nose so academics can read their own work, and makes it even harder for people with no academic affiliation, or an underresourced library, to access. This is not good for sociology or for academia, and it’s just not necessary. Getting the work out there, where colleagues and a broader audience can access it, isn’t that hard.

Many sociologists support greater openness. A fair number post their work on their own websites, or at Academia.edu, or elsewhere. But there is real value in having the work all in one place, and having that be a place that is committed to open science, rather than to monetizing your account.

By linking section awards to open access, ASA sections can help nudge sociology in this direction. Uploading to SocArXiv isn’t hard to do, but there’s an inertia factor to overcome. And since people want to win section awards, section award submissions are a good moment for overcoming it. If your paper is worth considering for an award, it should be worth sharing, and sections can help make this happen.

Making award-nominated papers open isn’t only good for the discipline, though. It’s good for the section, too. Having served on way too many section award committees in the last decade, I know that reading nominated papers is a great way to keep up with what’s going on in a subfield. This is often even more true of grad student submissions, which show you where the field is going. Why not get this great work out there sooner, and let people know the exciting things that are going on in your part of the discipline?

To sweeten the pot, SocArXiv is putting up $400 toward conference travel for the award winner of any “open” section award. We will also provide $250 of support for any individual award winner who uploaded their paper at the time they submitted to a nonparticipating section.

So if you think advancing openness is a good thing, and want to see your ASA sections support it, let them know. And if you have hesitations, bring them up in the comments — some we may be able to address, and we’d like to learn more about concerns we may not have anticipated.

(Curious what’s on SocArXiv? Here’s a few orgtheory relevant papers posted this summer:

• Pride and Prejudice: Employment Discrimination against Openly Gay Men in the United States, by András Tilcsik
• Do Millennials care about NPOs? Intergenerational Differences in Attitudes Towards Nonprofit Organizations, by Marko Kovic and Nina Hänsli
• Good to be Disliked? Exploring the Relationship Between Disapproval of Organizations and Job Satisfaction in the French Context, by Thomas Roulet
• Starting off on the Wrong Foot? Newly Founded Firms, TMT Structures, and the Unusualness Penalty, by Lisa Cohen and Heather Haveman
• Social Movements in Organizations, by Mikaila Mariel Lemonik Arthur

Want more details about what SocArXiv is? Click here. Or how award opening works? See this blog post. Or ask in the comments.)

why do some states have higher mass incarceration than others? comment on pamela oliver’s blog

Over at Race, Politics, and Justice, Pamela Oliver asks why her home state of Wisconsin has such high rates of Black imprisonment in comparison to other states, even in times when rates are falling:

Wisconsin has stayed at the top of the pile in Black incarceration even though its Black incarceration rate has been declining. How can this be? The answer is that all the other states have been declining faster. By putting a scatter plot of state imprisonment rates on consistent axes, I’ve been able to produce a really cool animation effect.  The data source is the\ National Corrections Reporting Program public “in custody” file. Rates are calculation on entire population (all ages). States voluntarily participate in this data collection program and appear and disappear from the plot depending on whether they reported for the appropriate year. States are also eliminated if more than 10% of their inmates are recorded as having unknown race. You’ll see if watch long enough that the relative positions of most states stay the same, but the whole distribution starts  moving downward (lower Black incarceration rates) and to the left (lower White incarceration rates) in the last few years. You may download both these images and explanatory material in PDF format  using this link.

Interesting. This is a classic example of the “dog that didn’t bark.” What happened in other states that did not happen in Wisconsin? A few hypotheses: Wisconsin reflects particularly bad conditions in segregated places like Milwaukee; fixed effects of prosecutors – Wisconsin district attorney’s are notoriously bad; police enforcement is unusually harsh. Add your hypotheses or explanation in the comments.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street 

bad reporting on bad science

This Guardian piece about bad incentives in science was getting a lot of Twitter mileage yesterday. “Cut-throat academia leads to natural selection of bad science,” the headline screams.

The article is reporting on a new paper by Paul Smaldino and Richard McElreath, and features quotes from the authors like, “As long as the incentives are in place that reward publishing novel, surprising results, often and in high-visibility journals above other, more nuanced aspects of science, shoddy practices that maximise one’s ability to do so will run rampant.”

Well. Can’t disagree with that.

But when I clicked through to read the journal article, the case didn’t seem nearly so strong. The article has two parts. The first is a review of review pieces published between 1962 and 2013 that examined the levels of statistical power reported in studies in a variety of academic fields. The second is a formal model of an evolutionary process through which incentives for publication quantity will drive the spread of low-quality methods (such as underpowered studies) that increase both productivity as well as the likelihood of false positives.

The formal model is kind of interesting, but just shows that the dynamics are plausible — something I (and everyone else in academia) was already pretty much convinced of. The headlines are really based on the first part of the paper, which purports to show that statistical power in the social and behavioral sciences hasn’t increased over the last fifty-plus years, despite repeated calls for it to do so.

Well, that part of the paper basically looks at all the papers that reviewed levels of statistical power in studies in a particular field, focusing especially on papers that reported small effect sizes. (The logic is that such small effects are not only most common in these fields, but also more likely to be false positives resulting from inadequate power.) There were 44 such reviews. The key point is that average reported statistical power has stayed stubbornly flat. The conclusion the authors draw is that bad methods are crowding out good ones, even though we know better, through some combination of poor incentives and selection that rewards researcher ignorance.

 

Statistical power has remained problematically low in the social and behavioral sciences

 

The problem is that the evidence presented in the paper is hardly strong support for this claim. This is not a random sample of papers in these fields, or anything like it. Nor is there other evidence to show that the reviewed papers are representative of papers in their fields more generally.

More damningly, though, the fields that are reviewed change rather dramatically over time. Nine of the first eleven studies (those before 1975) review papers from education or communications. The last eleven (those after 1995) include four from aviation, two from neuroscience, and one each from health psychology, software engineering, behavioral ecology, international business, and social and personality psychology. Why would we think that underpowering in the latter fields at all reflects what’s going on in the former fields in the last two decades? Maybe they’ve remained underpowered, maybe they haven’t. But statistical cultures across disciplines are wildly different. You just can’t generalize like that.

The news article goes on to paraphrase one of the authors as saying that “[s]ociology, economics, climate science and ecology” (in addition to psychology and biomedical science) are “other areas likely to be vulnerable to the propagation of bad practice.” But while these fields are singled out as particularly bad news, not one of the reviews covers the latter three fields (perhaps that’s why the phrasing is “other areas likely”?). And sociology, which had a single review in 1974, looks, ironically, surprisingly good — it’s that positive outlier in the graph above at 0.55. Guess that’s one benefit of using lots of secondary data and few experiments.

The killer is, I think the authors are pointing to a real and important problem here. I absolutely buy that the incentives are there to publish more — and equally important, cheaply — and that this undermines the quality of academic work. And I think that reviewing the reviews of statistical power, as this paper does, is worth doing, even if the fields being reviewed aren’t consistent over time. It’s also hard to untangle whether the authors actually said things that oversold the research or if the Guardian just reported it that way.

But at least in the way it’s covered here, this looks like a model of bad scientific practice, all right. Just not the kind of model that was intended.

[Edited: Smaldino points on Twitter to another paper that offers additional support for the claim that power hasn’t increased in psychology and cognitive neuroscience, at least.]

Appetite for Innovation: Creativity & Change at elBulli (To be published by Columbia University Press on July 12, 2016)

How is it possible for an organization to systematically enact changes in the larger system of which it is part? Using Ferran Adria’s iconic restaurant “elBulli” as an example of organizational creativity and radical innovation, Appetite for Innovation examines how Adria’s organization was able to systematically produce breakthroughs of knowledge within its field and, ultimately, to stabilize a new genre or paradigm in cuisine – the often called “experimental,” “molecular,” or “techno-emotional” culinary movement.

Recognized as the most influential restaurant in the world, elBulli has been at the forefront of the revolution that has inspired the gastronomic avant-garde worldwide. With a voracious appetite for innovation, year after year, Adrià and his team have broken through with new ingredients, combinations, culinary concepts and techniques that have transformed our way of understanding food and the development of creativity in haute cuisine.

Appetite for Innovation is an organizational study of the system of innovation behind Adrià’s successful organization. It reveals key mechanisms that explain the organization’s ability to continuously devise, implement and legitimate innovative ideas within its field and beyond. Based on exclusive access to meetings, observations, and interviews with renowned professionals of the contemporary gastronomic field, the book reveals how a culture for change was developed within the organization; how new communities were attracted to the organization’s work and helped to perpetuate its practice, and how the organization and its leader’s charisma and reputation were built and maintained over time. The book draws on examples from other fields, including art, science, music, theatre and literature to explore the research’s potential to inform practices of innovation and creativity in multiple kinds of organizations and industries.

The research for Appetite for Innovation was conducted when Adria’s organization was undergoing its most profound transformation, from a restaurant to a research center for innovation, “elBulli foundation”.  The book, therefore, takes advantage of this unique moment in time to retrace the story of a restaurant that became a legend and to explore underlying factors that led to its reinvention in 2011 into a seemingly unparalleled organizational model.

Appetite for Innovation is primarily intended to reach and be used by academic and professionals from the fields of innovation and organizations studies. It is also directed towards a non-specialist readership interested in the topics of innovation and creativity in general. In order to engage a wider audience and show the fascinating world of chefs and the inner-workings of high-end restaurants, the book is filled with photographs of dishes, creative processes and team’s dynamics within haute cuisine kitchens and culinary labs. It also includes numerous diagrams and graphs that illustrate the practices enacted by the elBulli organization to sustain innovation, and the networks of relationships that it developed over time. Each chapter opens with an iconic recipe created by elBulli as a way of illustrating the book’s central arguments and key turning points that enable the organization to gain a strategic position within its field and become successful.

To find a detailed description of the book please go to: http://cup.columbia.edu/book/appetite-for-innovation/9780231176781

Also, Forbes.com included Appetite for Innovation in its list of 17 books recommended for “creative leaders” to read this summer:  http://www.forbes.com/sites/berlinschoolofcreativeleadership/2016/05/15/17-summer-books-creative-leaders-can-read-at-the-beach/#7ac430985cef

 

tying our own noose with data? higher ed edition

I wanted to start this post with a dramatic question about whether some knowledge is too dangerous to pursue. The H-bomb is probably the archetypal example of this dilemma, and brings to mind Oppenheimer’s quotation of the Bhagavad Gita upon the detonation of Trinity: “Now I am become Death, the destroyer of worlds.

But really, that’s way too melodramatic for the example I have in mind, which is much more mundane. Much more bureaucratic. It’s less about knowledge that is too dangerous to pursue and more about blindness to the unanticipated — but not unanticipatable — consequences of some kinds of knowledge.

Maybe this wasn’t such a good idea.

The knowledge I have in mind is the student-unit record. See? I told you it was boring.

The student-unit record is simply a government record that tracks a specific student across multiple educational institutions and into the workforce. Right now, this does not exist for all college students.

There are records of students who apply for federal aid, and those can be tied to tax data down the road. This is what the Department of Education’s College Scorecard is based on: earnings 6-10 years after entry into a particular college. But this leaves out the 30% of students who don’t receive federal aid.

There are states with unit-record systems. Virginia’s is particularly strong: it follows students from Virginia high schools through enrollment in any not-for-profit Virginia college and then into the workforce as reflected in unemployment insurance records. But it loses students who enter or leave Virginia, which is presumably a considerable number.

But there’s currently no comprehensive federal student-unit record system. In fact at the moment creating one is actually illegal. It was banned in an amendment to the Higher Education Act reauthorization in 2008, largely because the higher ed lobby hates the idea.

Having student-unit records available would open up all kind of research possibilities. It would help us see the payoffs not just to college in general, but to specific colleges, or specific majors. It would help us disentangle the effects of the multiple institutions attended by the typical college student. It would allow us to think more precisely about when student loans do, and don’t, pay off. Academics and policy wonks have argued for it on just these grounds.

In fact, basically every social scientist I know would love to see student-unit records become available. And I get it. I really do. I’d like to know the answers to those questions, too.

But I’m really leery of student-unit records. Maybe not quite enough to stand up and say, This is a terrible idea and I totally oppose it. Because I also see the potential benefits. But leery enough to want to point out the consequences that seem likely to follow a student-unit record system. Because I think some of the same people who really love the idea of having this data available would be less enthused about the kind of world it might help, in some marginal way, create.

So, with that as background, here are three things I’d like to see data enthusiasts really think about before jumping on this bandwagon.

First, it is a short path from data to governance. For researchers, the point of student-level data is to provide new insights into what’s working and what isn’t: to better understand what the effects of higher education, and the financial aid that makes it possible, actually are.

But for policy types, the main point is accountability. The main point of collecting student-level data is to force colleges to take responsibility for the eventual labor market outcomes of their students.

Sometimes, that’s phrased more neutrally as “transparency”. But then it’s quickly tied to proposals to “directly tie financial aid availability to institutional performance” and called “an essential tool in quality assurance.”

Now, I am not suggesting that higher education institutions should be free to just take all the federal money they can get and do whatever the heck they want with it. But I am very skeptical that, in general, the net effect of accountability schemes is generally positive. They add bureaucracy, they create new measures to game, and the behaviors they actually encourage tend to be remote from the behaviors they are intended to encourage.

Could there be some positive value in cutting off aid to institutions with truly terrible outcomes? Absolutely. But what makes us think that we’ll end up with that system, versus, say, one that incentivizes schools to maximize students’ earnings, with all the bad behavior that might entail? Anyone who seriously thinks that we would use more comprehensive data to actually improve governance of higher ed should take a long hard look at what’s going on in the UK these days.

Second, student-unit records will intensify our already strong focus on the economic return to college, and further devalue other benefits. Education does many things for people. Helping them earn more money is an important one of those things. It is not, however, the only one.

Education expands people’s minds. It gives them tools for taking advantage of opportunities that present themselves. It gives them options. It helps them to find work they find meaningful, in workplaces where they are treated with respect. And yes, selection effects — or maybe it’s just because they’re richer — but college graduates are happier and healthier than nongraduates.

The thing is, all these noneconomic benefits are difficult to measure. We have no administrative data that tracks people’s happiness, or their health, let alone whether higher education has expanded their internal life.

What we’ve got is the big two: death and taxes. And while it might be nice to know whether today’s 30-year-old graduates are outliving their nongraduate peers in 50 years, in reality it’s tax data we’ll focus on. What’s the economic return to college, by type of student, by institution, by major? And that will drive the conversation even more than it already does. Which to my mind is already too much.

Third, social scientists are occupationally prone to overestimate the practical benefit of more data. Are there things we would learn from student-unit records that we don’t know? Of course. There are all kinds of natural experiments, regression discontinuities, and instrumental variables that could be exploited, particularly around financial aid questions. And it would be great to be able to distinguish between the effects of “college” and the effects of that major at this college.

But we all realize that a lot of the benefit of “college” isn’t a treatment effect. It’s either selection — you were a better student going in, or from a better-off family — or signal — you’re the kind of person who can make it through college; what you did there is really secondary.

Proposals to use income data to understand the effects of college assume that we can adjust for the selection effects, at least, through some kind of value-added model, for example. But this is pretty sketchy. I mean, it might provide some insights for us to think about. But as a basis for concluding that Caltech, Colgate, MIT, and Rose-Hulman Institute of Technology (the top five on Brookings’ list) provide the most value — versus that they have select students who are distinctive in ways that aren’t reflected by adjusting for race, gender, age, financial aid status, and SAT scores — is a little ridiculous.

So, yeah. I want more information about the real impact of college, too. But I just don’t see the evidence out there that having more information is going to lead to policy improvements.

If there weren’t such clear potential negative consequences, I’d say sure, try, it’s worth learning more even if we can’t figure out how to use it effectively. But in a case where there are very clear paths to using this kind of information in ways that are detrimental to higher education, I’d like to see a little more careful thinking about the real likely impacts of student-unit records versus the ones in our technocratic fantasies.

design-focused review: a guest post by samuel r. lucas

Samuel R. Lucas is professor of sociology at the University of California, Berkeley. He works on education, social mobility, and research methods. This guest post proposes a reform of the journal review process.

On-going discussion about the journal publication process is laudable. I support many of the changes that have been suggested, such as the proposal to move to triple-blind review, and implemented, such as the rise of new journals that reject “dictatorial revi”–oops, I mean “developmental review.” I suggest, however, that part of the problem is that reviewers are encouraged to weigh in on anything–literally anything! I’ve reviewed papers and later received others’ reviews only to find a reviewer ignored almost all of the paper, weighing in on such issues as punctuation and choice of abbreviations for some technical terms. Although such off-point reviews are rare, they indicate that reviewers perceive it legitimate to weigh in on anything and everything. But a system allowing unlimited bases of review is part of the problem with peer review, for it shifts too much power to reviewers while at the same time providing insufficient guidance on what will be helpful in peer review. I contend that we need dispense with our kitchen-sink reviewing system by removing from reviewer consideration two aspects of papers: framing and findings.

Framing is a matter of taste and, as there is no accounting for taste, framing offers fertile ground for years of delay. Framing is an easy way to hold a paper hostage, because most solid papers could be framed in any one of several ways, and often multiple frames are equally valid. Authors should be allowed to frame their work as they see fit, not be forced to alter the frame because a reviewer reads the paper differently than the author. A reviewer who feels a paper should be framed differently should wait for its publication and then submit a paper that notes that the paper addressed Z but missed its connection to Q. Such an approach would make any worthwhile debate on framing public while freeing authors to place their ideas into the dialogue as well.

As for findings, peer review should be built on the following premise: if you accept the methods, then you accept the findings enough for the paper to enter the peer-reviewed literature. Thus, reviewers should assess whether the paper’s (statistical, experimental, qualitative) research design can answer the paper’s research question, but not the findings produced by the solid research design. Allowing reviewers to evaluate findings allows reviewers to (perhaps inadvertantly) scrutinize papers differently depending on the findings. To prevent such possibilities, journals should allow authors to request a findings-embargoed review, for which the journal would remove the findings section of the paper as well as the findings from: 1)the abstract, and, 2)the discussion/conclusion section of the paper before delivering the paper for review. As some reviewers may regard reading soon-to-be-published work early as a benefit of reviewing, reviewers could be sent full manuscripts if the paper is accepted for publication.

A review system in which reviewers do not review framing and findings is a design-focused review system. Once a paper passes a design-focused review, editors can conduct an in-house assessment to assure findings are accurately conveyed and the framing is coherent. The editors, unlike reviewers, see the population of submissions, and thus, unlike reviewers, are well-placed to fairly and consistently assess any other issues. Editors will be even more enabled to make such calls if they can make them only for the papers reviewers have determined satisfy the basic criterion of having a design solid enough to answer the question the paper poses.

The current kitchen-sink review system has become increasingly time-consuming and perhaps capricious, hardly positive features for effective peer review. If findings were embargoed and reviewers were discouraged from treating their preferred frame as essential to a quality paper, review times could be chopped dramatically and revise and resubmit processes would be focused on solidifying design. As a result, design-focused review could lower our collective workload by reducing the number of taste-driven rounds of review we experience as authors and reviewers, while simultaneously reducing authors’ potentially paralyzing concern that mere matters of taste will block their research from timely publication. Design-focused review may thus make peer review work better for everyone.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

how scientists can help us avoid the next flint

In a story full of neglect and willful ignorance, there are a few heroes. One is Mona Hanna-Attisha, the Flint pediatrician and Michigan State professor who raised the alarm with data on kids’ blood-lead levels from the local hospital. Another is Marc Edwards, the Virginia Tech environmental engineer who took on the Michigan Department of Environmental Quality after a Flint resident sent him a lead-rich water sample for testing.

Hanna-Attisha and Edwards provide shining examples of how academics can use science to hold the powers-that-be accountable and make meaningful change.

Taking on the status quo is hard. But as Edwards discusses in the Chronicle, it’s becoming ever-harder to do that from within universities:

I am very concerned about the culture of academia in this country and the perverse incentives that are given to young faculty. The pressures to get funding are just extraordinary. We’re all on this hedonistic treadmill — pursuing funding, pursuing fame, pursuing h-index — and the idea of science as a public good is being lost….What faculty person out there is going to take on their state, the Michigan Department of Environmental Quality, and the U.S. Environmental Protection Agency?…When was the last time you heard anyone in academia publicly criticize a funding agency, no matter how outrageous their behavior? We just don’t do these things….Everyone’s invested in just cranking out more crap papers.

When faculty defend academic freedom, tenure is often the focus. And certainly tenure provides one kind of protection for scientists like Hanna-Attisha (though she doesn’t yet have it) or Edwards who want to piss off the powerful.

But as this interview — and you should really read the whole thing — makes clear, tenure isn’t the only element of the academic ecosystem that allows people to speak out. Scientists can’t do their work without research funding, or access to data. When funders have interests — whether directly economic, as when oil and gas companies fund research on the environmental impacts of fracking, or more organizational, as when environmental agencies just don’t want to rock the boat — that affects what scientists can do.

So in addition to tenure, a funding ecosystem that includes multiple potential sources and that excludes the most egregiously self-interested will encourage independent science.

But beyond that, we need to defend strong professional cultures. Hanna-Attisha emphasizes how the values of medicine both motivated her (“[T]his is what matters. This is what we do … This is why we’re here”) and prompted her boss’s support (“Kids’ health comes first”), despite the “politically messy situation” that might have encouraged the hospital’s silence. Edwards lectures his colleagues about “their obligation as civil engineers to protect the public” and says, “I didn’t get in this field to stand by and let science be used to poison little kids.”

Intense economic pressures, though, make it hard to protect such this kind of idealism. As market and financial logics come to dominate institutions like hospitals and universities, professional values gradually erode. It takes a concerted effort to defend them when everything else encourages you to keep your head down and leave well enough alone.

Promoting academic independence isn’t without its downsides. Scientists can become solipsistic, valuing internal status over real-world impact and complacently expecting government support as their due. The balance between preserving a robust and independent academic sector and ensuring scientists remain accountable to the public is a delicate one.

But if I have to choose between two risks—that science might be a bit insular and too focused on internal incentives, or that the only supporters of science have a one-sided interest in how the results turn out—I’ll take the first one every time.

that chocolate milk study: can we blame the media?

A specific brand of high-protein chocolate milk improved the cognitive function of high school football players with concussions. At least that’s what a press release from the University of Maryland claimed a few weeks ago. It also quoted the superintendent of the Washington County Public Schools as saying, “Now that we understand the findings of this study, we are determined to provide Fifth Quarter Fresh [the milk brand] to all of our athletes.”

The problem is that the “study” was not only funded in part by the milk producer, but is unpublished, unavailable to the public and, based on the press release — all the info we’ve got — raises immediate methodological questions. Certainly there are no grounds for making claims about this milk in particular, since the control group was given no milk at all.

The summary also raises questions about the sample size. The total sample included 474 high school football players, but included both concussed and non-concussed players. How many of these got concussions during one season? I would hope not enough to provide statistical power — this NAS report suggests high schoolers get 11 concussions per 10,000 football games and practices.

And even if the sample size is sufficient, it’s not clear that the results are meaningful. The press release suggests concussed athletes who drank the milk did significantly better on four of thirty-six possible measures — anyone want to take bets on the p-value cutoff?

Maryland put out the press release nearly four weeks ago. Since then there’s been a slow build of attention, starting with a takedown by Health News Review on January 5, before the story was picked up by a handful of news outlets and, this weekend, by Vox. In the meanwhile, the university says in fairly vague terms that it’s launched a review of the study, but the press release is still on the university website, and similarly questionable releases (“The magic formula for the ultimate sports recovery drink starts with cows, runs through the University of Maryland and ends with capitalism” — you can’t make this stuff up!) are up as well.

Whoever at the university decided to put out this press release should face consequences, and I’m really glad there are journalists out there holding the university’s feet to the fire. But while the university certainly bears responsibility for the poor decision to go out there and shill for a sponsor in the name of science, it’s worth noting that this is only half of the story.

There’s a lot of talk in academia these days about the status of scientific knowledge — about replicability, bias, and bad incentives, and how much we know that “just ain’t so.” And there’s plenty of blame to go around.

But in our focus on universities’ challenges in producing scientific knowledge, sometimes we underplay the role of another set of institutions: the media. Yes, there’s a literature on science communication that looks as the media as intermediary between science and the public. But a lot of it takes a cognitive angle on audience reception, and it’s got a heavy bent toward controversial science, like climate change or fracking.

More attention to media as a field, though, with rapidly changing conditions of production, professional norms and pathways, and career incentives, could really shed some light on the dynamics of knowledge production more generally. It would be a mistake to look back to some idealized era in which unbiased but hard-hitting reporters left no stone unturned in their pursuit of the public interest. But the acceleration of the news cycle, the decline of journalism as a viable career, the impact of social media on news production, and the instant feedback on pageviews and clickthroughs all tend to reinforce a certain breathless attention to the latest overhyped university press release.

It’s not the best research that gets picked up, but the sexy, the counterintuitive, and the clickbait-ish. Female-named hurricanes kill more than male hurricanes. (No.) Talking to a gay canvasser makes people support gay marriage. (Really no.) Around the world, children in religious households are less altruistic than children of atheists. (No idea, but I have my doubts.)

This kind of coverage not only shapes what the public believes, but it shapes incentives in academia as well. After all, the University of Maryland is putting out these press releases because it perceives it will benefit, either from the perception it is having a public impact, or from the goodwill the attention generates with Fifth Quarter Fresh and other donors. Researchers, in turn, will be similarly incentivized to focus on the sexy topic, or at least the sexy framing of the ordinary topic. And none of this contributes to the cumulative production of knowledge that we are, in theory, still pursuing.

None of this is meant to shift the blame for the challenges faced by science from the academic ecosystem to the realm of media. But if you really want to understand why it’s so hard to make scientific institutions work, you can’t ignore the role of media in producing acceptance of knowledge, or the rapidity with which that role is changing.

After all, if academics themselves can’t resist the urge to favor the counterintuitive over the mundane, we can hardly blame journalists for doing the same.

why do universities salivate over money-losing grants?

Happy new year. Guess what my New Year’s resolution is. To that end, a few quick thoughts on universities and the grant economy to dip a toe back in the water.

We all know that American universities (well, not only American universities) are increasingly hungry for grants. When state funding stagnates, and tuition revenues are limited by politics or discounting, universities look to their faculty to bring in money through grants. Although this may be a zero-sum game across universities (assuming total funding is fixed), it is unsurprising that administrations would intensify grant-seeking when faced with tight budgets.

Of course, it’s only unsurprising if grants actually make money for the university. But a variety of observers, from the critical to the self-interested, have argued that the indirect costs that many grants bring in – the part that pays not for the direct cost of research, but for overhead expenses like keeping the network running, the library open, and the heat and electricity on – don’t actually cover the full expense of conducting research.

Instead, they suggest that every grant the university brings in costs it another 9% or so in unreimbursed overhead. In addition, about 12% of total research spending consists of universities spending their own money on research. While some of this goes to support work unlikely to receive external funding (e.g. research in the humanities), I think it’s safe to assume that most of it is related to the search for external grants – it’s seed funding for projects with the potential for external funding, or bridge funding for lab faculty between grants. (These numbers come from the Council on Government Relations, a lobbying organization of research universities.)

If that’s the case, it means that when faculty bring in grants, even federal grants that come with an extra 50% or so to pay for overhead costs, it costs the university money. Money that could be spent on instruction, or facility maintenance, or even on research itself. So how can we make sense of the fact that universities are intensifying their search for grants, even as the numbers suggest that grants cost universities more they gain them?

I can think of at least three reasons this might be the case:

1.  The numbers are wrong.

It is notoriously difficult to estimate the “real” indirect costs of research. How much of the library should your grant pay for? How much of the heat, if it’s basically supporting a grad student who would be sitting in the same shared office with or without the grant? There are conventions here, but they are just that – conventions. And maybe universities have a better sense of the “real” costs, which might be lower than standard accounting would suggest. COGR has an interest in making research look expensive, so government is generous about covering indirect costs. And critics of the university (with whom I sympathize) have a different interest in highlighting the costs of research, since they see a heavy grants focus as coming at the cost of education and of the humanities and social sciences. (See e.g. this recent piece by Chris Newfield, which inspired the line of thought behind this post.)

Certainly the numbers are squishy, and the evidence that grant-seeking costs universities more than it gains them isn’t airtight. But I haven’t seen anyone make a strong case that universities are actually making money from indirect costs. So I’m skeptical that these numbers are out-and-out wrong, although open to better evidence.

2. It’s basically political and/or symbolic, not financial.

A second possibility is that the additional dollars aren’t really the point. The point is that universities exist in a status economy in which having a large research enterprise is integral to many forms of success, from attracting desirable faculty and students, to appearing in a positive light to politicians (more relevant for public than private universities), to attracting donations from those who want to give to an institution that is among the “best”. Or, in a slight variation, maybe the perceived political benefits of having a large grant apparatus – of being on the cutting edge of science, of being seen as economically valuable – is seen as outweighing any extra costs. After all, what’s an extra 10% per grant if it makes the difference between the state increasing or cutting your appropriations over the next decade? (Again, most relevant for publics.)

These dynamics are real, but they don’t explain the intensification of the search for grants in response to tight budgets, except insofar as tight budgets also intensify the status competition. But it really seems to me that administrators see grants as a direct financial solution, not an indirect one. So I think that symbolic politics is a piece of the puzzle, but not the only one.

3.  Not all dollars are created equal.

Different dollars have different values to different people. Academic scientists often like industry grants because they tend to be more flexible than government money. Administrators, on the other hand, don’t, since such grants typically don’t cover overhead expenses.

Perhaps something related is going on with the broader search for grants. Maybe, even if grants really do cost more than they bring in for universities, administrators don’t perceive the revenues and the expenses in parallel ways. After all, those indirect costs provide identifiable extra dollars the university wouldn’t have seen otherwise. But the “excess” expenses are sort of invisible. The university is going to pay for the heat and the library either way; even if you know the research infrastructure has to be supported, you might assume that the marginal overhead cost of an additional grant doesn’t make that much difference. (Maybe you’d even be right.) And people might not see some costs – like university seed funding for potentially fundable research – as an expense of grant-seeking, even if that’s why they exist.

I think this is probably a big part of the explanation. The extra revenues of grants are visible and salient; the extra costs are hidden and easy to discount. So, rightly or wrongly, administrators turn to grant-seeking in tight times despite the fact that it actually costs universities money.

There are some other possibilities I’m not considering here. For example, maybe this is about the interests of different specific groups within the organization – e.g. about competitions among deans, or between upper administration and trustees. But I think #2 and #3 capture a lot of what’s going on.

So, if you think this dynamic (the intensification of grant-seeking) is kind of dysfunctional, what do you do? Well, pointing out how much research really costs the university – loudly and repeatedly – is probably a good idea. Make those “extra” costs as visible and salient as the revenues. (Though it would be SO NICE if the numbers were better.)

But don’t discount #2 – even if any extra costs of grants are made clear, universities aren’t going to give up the search for them. Because while the money grants bring in matters, they also have value as status capital, and that outweighs any unreimbursed costs they incur. Grants may not quite cover those pesky infrastructure costs. But the legitimacy they collectively confer is, quite literally, priceless.

new book Handbook of Qualitative Organizational Research Innovative Pathways and Methods (2015, Routledge) now available

At orgtheory, we’ve had on-going discussions about how to undertake research.  For example, I’ve shared my own take on dealing with the IRB, gaining access to organizations, undertaking ethnography , timing and pacing research, writing for wider audiences, and what is ethnography good for?  Guest blogger Ellen Berrey elaborated her thoughts on how to get access to organizations, and we’ve had at least three discussions about the challenges of anonymizing names and identities of persons and organizations, including guest blogger Victor Tan Chen’s post, guest blogger Ellen Berrey’s post, and Fabio’s most recent post here.

Looking for more viewpoints about how to undertake organizational research?  Preparing a research proposal?  Need a new guide for a methods or organizations class?  Rod Kramer and Kim Elsbach have co-edited the Handbook of Qualitative Organizational Research Innovative Pathways and Methods (2015, Routledge). 

In the introduction, Kramer and Elsbach describe the impetus for the volume:

There were several sources of inspiration that motivated this volume. First and foremost was a thoughtful and provocative article by Jean Bartunek, Sara Rynes, and Duane Ireland that appeared in the Academy of Management Journal in 2006. This article published a list of the 17 most interesting organizational papers published in the last 100 years. These papers were identified by Academy of Management Journal board members—all of whom are leading organizational scholars cognizant of  the best work being done in their respective areas. A total of 67 board members nominated 160 articles as exceptionally interesting; those articles that received two or more nominations were deemed the most interesting. Of these exceptional articles, 12 (71%) involved qualitative methods.

This result strongly mirrors our own experience as organizational researchers. Although both of us have used a variety of methods in our organizational research (ranging from experimental lab studies and surveys to computer-based, agent simulations), our favorite studies by far have been our qualitative studies (including those we have done together). One of the qualities we have come to most appreciate, even cherish, about qualitative research is the sense of discovery and the opportunity for genuine intellectual surprise. Rather than merely seeking to confirm a preordained hypothesis or “nail down” an extrapolation drawn from the extant literature, our inductive studies, we found, invariably opened up exciting, unexpected intellectual doors and pointed us toward fruitful empirical paths for further investigation. In short, if life is largely all about the journey rather than destination, as the adage asserts, we’ve found qualitative research most often gave us a road we wanted to follow.

stuff that doesn’t replicate

Here’s the list (so far):

• Economics
• Sociology
• Psychology
• Cancer research
• Genetics

Some people might want to hand wave the problem away or jump to the conclusion that science is broken. There’s a more intuitive explanation – science is “brittle.” That is, once you get past some basic and important findings, you get to findings that are small in size, require many technical assumptions, or rely on very specific laboratory/data collection conditions.

There should be two responses. First, editors should reject submissions which might depend on “local conditions” or very small results or send them to lower tier journals. Second, other researchers should feel free to try to replicate research. This is appropriate work for early career academics who need to learn how work is done. Of course, people who publish in top journals, or obtain famous results, should expect replication requests.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

movements and inhabited institutions: the case of latino student groups

A key insight from research on student activism is that the college environment has a strong influence on how that activism expresses itself. We saw that in Amy Binder and Kate Wood’s study of conservative groups. Daisy Reyes has an article in Sociology of Education that explores this issue with Latino groups and links it to institutional theory:

To comply with ideals of multiculturalism and diversity, postsecondary institutions incorporate Latino students into distinct campus cultures. These cultures influence how students interact with one another, the university community at large, and communities outside of campus, ultimately shaping how students inhabit Latino politics. Drawing on data from 20 months of ethnographic fieldwork with six student organizations and 60 in-depth interviews, I compare Latino student organizations in a liberal arts college, a research university, and a regional public university. Building on inhabited institutional theory, I identify dimensions of campus cultures that work in interaction with students to produce three divergent forms of ethnic political expression: deliberative, divisive, and contentious. Inhabited institutionalism helps explain why Latino politics takes distinct forms in specific academic contexts and suggests that strong collegiate incorporation may paradoxically serve to suppress Latino student engagement in political activism outside the campus gates.

Read the entire article here. Recommended.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

inside higher education discusses replication in psychology and sociology

Science just published a piece showing that only a third of articles from major psychology journals can be replicated. That is, if you reran the experiments, only a third of experiments will have statistically significant results. The details of the studies matter as well. The higher the p-value, the less like you were to replicate and “flashy” results were less likely to replicate.

Insider Education spoke to me and other sociologists about the replication issue in our discipline. A major issue is that there is no incentive to actually assess research since it seems to be nearly impossible to publish replications and statistical criticisms in our major journals:

Recent research controversies in sociology also have brought replication concerns to the fore. Andrew Gelman, a professor of statistics and political science at Columbia University, for example, recently published a paper about the difficulty of pointing out possible statistical errors in a study published in the American Sociological Review. A field experiment at Stanford University suggested that only 15 of 53 authors contacted were able or willing to provide a replication package for their research. And the recent controversy over the star sociologist Alice Goffman, now an assistant professor at the University of Wisconsin at Madison, regarding the validity of her research studying youths in inner-city Philadelphia lingers — in part because she said she destroyed some of her research to protect her subjects.

Philip Cohen, a professor of sociology at the University of Maryland, recently wrote a personal blog post similar to Gelman’s, saying how hard it is to publish articles that question other research. (Cohen was trying to respond to Goffman’s work in the American Sociological Review.)

“Goffman included a survey with her ethnographic study, which in theory could have been replicable,” Cohen said via email. “If we could compare her research site to other populations by using her survey data, we could have learned something more about how common the problems and situations she discussed actually are. That would help evaluate the veracity of her research. But the survey was not reported in such a way as to permit a meaningful interpretation or replication. As a result, her research has much less reach or generalizability, because we don’t know how unique her experience was.”

Readers can judge whether Gelman’s or Cohen’s critiques are correct. But the broader issue is serious. Sociology journals simply aren’t publishing error correction or replication, with the honorable exception of Sociological Science which published a replication/critique of the Brooks/Manza (2006) ASR article. For now, debate on the technical merits of particular research seems to be the purview of blog posts and book reviews that are quickly forgotten. That’s not good.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

sociologists need to be better at replication – a guest post by cristobal young

Cristobal Young is an assistant professor at Stanford’s Department of Sociology. He works on quantitative methods, stratification, and economic sociology. In this post co-authored with Aaron Horvath, he reports on the attempt to replicate 53 sociological studies. Spoiler: we need to do better.

Do Sociologists Release Their Data and Code? Disappointing Results from a Field Experiment on Replication.

 

Replication packages – releasing the complete data and code for a published article – are a growing currency in 21^st century social science, and for good reasons. Replication packages help to spread methodological innovations, facilitate understanding of methods, and show confidence in findings. Yet, we found that few sociologists are willing or able to share the exact details of their analysis.

We conducted a small field experiment as part of a graduate course in statistical analysis. Students selected sociological articles that they admired and wanted to learn from, and asked the authors for a replication package.

Out of the 53 sociologists contacted, only 15 of the authors (28 percent) provided a replication package. This is a missed opportunity for the learning and development of new sociologists, as well as an unfortunate marker of the state of open science within our field.

Some 19 percent of authors never replied to repeated requests, or first replied but never provided a package. More than half (56 percent) directly refused to release their data and code. Sometimes there were good reasons. Twelve authors (23 percent) cited legal or IRB limitations on their ability to share their data. But only one of these authors provided the statistical code to show how the confidential data were analyzed.

Why So Little Response?

A common reason for not releasing a replication package was because the author had lost the data – often due to reported computer/hard drive malfunctions. As well, many authors said they were too busy or felt that providing a replication package would be too complicated. One author said they had never heard of a replication package. The solutions here are simple: compiling a replication package should be part of a journal article’s final copy-editing and page-proofing process.

More troubling is that a few authors openly rejected the principle of replication, saying in effect, “read the paper and figure it out yourself.” One articulated a deep opposition, on the grounds that replication packages break down the “barriers to entry” that protect researchers from scrutiny and intellectual competition from others.

The Case for Higher Standards

Methodology sections of research articles are, by necessity, broad and abstract descriptions of their procedures. However, in most quantitative analyses, the exact methods and code are on the author’s computer. Readers should be able to download and run replication packages as easily as they can download and read published articles. The methodology section should not be a “barrier to entry,” but rather an on-ramp to an open and shared scholarly enterprise.

When authors released replication packages, it was enlightening for students to look “under the hood” on research they admired, and see exactly how results were produced. Students finished the process with deeper understanding of – and greater confidence in – the research. Replication packages also serve as a research accelerator: their transparency instills practical insight and confidence – bridging the gap between chalkboard statistics and actual cutting-edge research – and invites younger scholars to build on the shoulders of success. As Gary King has emphasized, replications have become first publications for many students, and helped launched many careers – all while ramping up citations to the original articles.

In our small sample, little more than a quarter of sociologists released their data and code. Top journals in political science and economics now require on-line replication packages. Transparency is no less crucial in sociology for the accumulation of knowledge, methods, and capabilities among young scholars. Sociologists – and ultimately, sociology journals – should embrace replication packages as part of the lasting contribution of their research.

Table 1. Response to Replication Request

Response	Frequency	Percent
Yes:   Released data and code for paper	15	28%
No: Did not release	38	72%
Reasons for “No”
IRB / legal / confidentiality issue	12	23%
No response / no follow up	10	19%
Don’t have data	6	11%
Don’t have time / too complicated	6	11%
Still using the data	2	4%
‘See the article and figure it out’	2	4%
Total	53	100%

Note: For replication and transparency, a blinded copy of the data is available on-line. Each author’s identity is blinded, but the journal name, year of publication, and response code is available. Half of the requests addressed articles in the top three journals, and more than half were published in the last three years.

Figure 1: Illustrative Quotes from Student Correspondence with Authors:

Positive:

1. “Here is the data file and Stata .do file to reproduce [the] Tables….  Let me know if you have any questions.”
2. “[Attached are] data and R code that does all regression models in the paper. Assuming that you know R, you could literally redo the entire paper in a few minutes.”

Negative:

3. “While I applaud your efforts to replicate my research, the best guidance I can offer
   is that the details about the data and analysis strategies are in the paper.”
4. “I don’t keep or produce ‘replication packages’… Data takes a significant amount of human capital and financial resources, and serves as a barrier-to-entry against other researchers… they can do it themselves.”

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

party in the streeet: response to econlog commenters

Last week, Bryan Caplan wrote two lengthy posts about Party in the Street (here and here). He focuses on a few issues: the differences between Republican and Democratic administrations on war policy and the exaggeration of differences by activists. Bryan also argues that the arguments typically made by peace activists aren’t those he would make. Rather than condemn specific politicians or make blanket statements about war, he focuses on the death of innocents and war’s unpredictability (e.g., it is hard to judge if wars work ex ante).

The commenters raised a number of questions and issues. Here are a few:

• Jacob Geller asks whether the collapse of the peace movement is spurious and could be attributed to other factors (e.g., the economy). Answer: There are multiple ways to assess this claim – the movement began its slide pre-recession (true), partisans are more likely to disappear than non-partisans during the recession (true), and the movement did not revive post-recession (true – e.g., few democrats have protested Obama’s war policies). Movements rise and fall for many reasons, but in this case, partisanship is almost certainly a factor.
• Michael suggested that there was a Democratic war policy difference in that Al Gore would not have fought Iraq. One can’t establish anything with certainty using counter factual history, but Frank Harvey suggested that President Gore would like have fought Iraq, given the long standing enmity and low level armed conflict between Iraq and the Clinton administration  (including Gore).
• Also, a few people raised the issue of voting and if the antiwar issue was salient for Democrats. A few comments – one is that in data about activists, Democrats tended to view Obama’s management of war in better terms than non-partisans. Another point is that opinions on the war affected vote choice in multiple elections. The issue, though, isn’t whether Democrats were motivated by their attitudes on the Iraq War. The issue is how that is linked to movement participation and how that changes over time, given electoral events. All evidence suggests that the democratic party and the antiwar movement dissociated over time, leading to the peace movement’s collapse.

Thanks for the comments!

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

let’s just burn 20% of our research dollars

Plummeting grant funding rates are back in the news, this time in the U.K., where success rates in the Economic and Social Research Council—a rough equivalent to NSF’s SBE division—have dropped to 13%. In sociology, it’s even lower—only 8% of applications were funded in 2014-15.

I’ve written before about the waste of resources associated with low funding rates. But this latest round prompted me to do some back-of-the-envelope calculations. Disclaimer: these numbers are total guesses based on my experience in the U.S. system. I think they are pretty conservative. But I would love to see more formal estimates.

book spotlight: remaking college: the changing ecology of higher education, edited by kirst and stevens

Recent orgtheory posts excepted, we pay way too much attention to a tiny handful of higher education institutions in the U.S. (Not to mention too much attention to the U.S. relative to the rest of the world.)

Academic chatter often assumes research universities are the prototypical higher ed organization, even though only 23% of students are enrolled in such universities (RU/VH or RU/H). By comparison, more than a third are enrolled in community colleges, and nearly 10% in for-profit institutions.

At the level of public attention, focus gets even narrower. A New York Times search gets 310 hits for “community college,” versus nearly 13,000 for “Harvard.” Recently historian David Perry surveyed two months of NYT op-eds containing the word “professor” and found

zero by community college or lower-status teaching school profs, zero by branch campus public profs, and a handful by top liberal arts schools (Smith, Dickinson) or lower-tier R1 publics (Colorado State, South Carolina).

And of course nothing gets our collective hearts aflutter like a good old fashioned spat over whether the Ivy League is an awful, awful place.

So kudos to Michael Kirst and Mitchell Stevens for noticing that the world of higher ed is bigger than that. Remaking College: The Changing Ecology of Higher Education, published a couple of months ago by Stanford UP, focuses on the institutions that are underappreciated by the media and scholars: comprehensive colleges, community colleges, for-profit colleges. By bringing together a diverse group of academics — several of whom take an explicitly organizational approach — to focus on broad-access institutions, they have done the field a real service.

The essays cover a range of ground and approaches. Several, including an orienting one by W. Richard Scott, conceptualize higher ed as an ecology or field. I’ll just highlight a couple I particularly enjoyed here.

In “The Classification of Organizational Forms: Theory and Application to the Field of Higher Education,” Martin Ruef and Manish Nag use topic models based on IPEDS data to generate new sets of categories for U.S. postsecondary institutions. From mission statements, for example, they infer not only two distinct clusters of liberal arts schools and two of community colleges, but several additional types of institutions — globally-oriented colleges, Christian colleges, medical tech schools, student-oriented universities — that might otherwise go unnoticed. Like other good work that identifies patterns from texts, it prompts a rethinking of cultural identity beyond assumed categories.

Regina Deil-Amen makes a significant contribution just by hammering home how atypical the “typical” college student really is. Nearly three-quarters of first-year undergraduates are enrolled in community colleges or for-profit institutions. 53% are not enrolled full-time. Only 13% live on campus. 13 percent! Her quotes of interviews with lower-income and Latino students, who are dealing with family stresses and financial struggles, are telling:

My family has a lot of financial problems, so that’s another stress that I’m constantly dealing with. I have to call them like, ‘Mom, are you gonna be able to pay rent this month?’…I’ve actually used some of my loans to help them pay their rent this year. (p. 146)

These firsthand accounts reinforce how inaccurate the picture of a dependent 18-year-old striking out on her own for the first time actually is.

I also enjoyed Richard Arum and Josipa Roksa’s reflection on measuring college performance, where they emphasize that they

have vehemently argued against the desirability of an externally imposed accountability schema. We are deeply skeptical of increased centralized regulation of this character—fearing that the unintended consequences would far outweigh any benefits—and have instead called for institutions themselves to assume enhance responsibility for monitoring and improving student outcomes. (p. 170)

I’m not sure they know how to measure college quality either, but it’s a thoughtful piece.

Higher ed really is a diverse organizational ecology, and it’s going to take a lot of work to map out the whole landscape. But I’m very glad that people like Kirst and Stevens are moving us in that direction.

teaching, tenure, and academic freedom

As events continue to unfold in Wisconsin, defenses of tenure are popping up in various places. For the most part, these are focused on how weakening tenure would 1) limit academic freedom, 2) drive faculty to other universities, and 3) subject them to political reprisals.

These are all true. One only has to think about climate research, or UNC’s Poverty Center, to realize that the threat to academic freedom is very real.

What is less clear is why the public should care. Sure, some will. But lots of people believe climate science is corrupt, and that centers like UNC’s are inappropriately political. Any good defense of the public university—of tenure within it or support for it more generally—has to appeal to a broad swath of people.

I suggested the other day that the business community cares about science, and that that is one potential source of support for higher ed, at least, if not necessarily for tenure. But what the average American cares about most with regard to universities is not science, but teaching.

And here…crickets.

Clay Shirky argued at Crooked Timber that in fact professors don’t do very much teaching, and when the public learns this they will revolt. I think he sees the world too much through the lens of NYU, and that if you look at the higher ed field as a whole, there is lots of teaching going on, including by tenure-track faculty.

But where he is right is that what most people outside higher ed care about is not research, but teaching. Fortunately, there are strong arguments to be made that link tenure and teaching quality. For example, Mikaila pointed out in the comments that

performance funding initiatives which emphasize on-time graduation rates would tend to encourage a decrease in academic rigor so that students make adequate academic progress and do not fail or withdraw from courses–something we could easily achieve by giving our students open-book fill-in-the-blank tests with As for all. It is tenure which protects us from such a demand and thus tenure that gives us the best chance of ensuring that students have the opportunity to receive a high-quality, rigorous education that challenges them and helps them learn and develop the skills which will benefit them economically, socially, culturally, and personally for the rest of their lives.

These are the kinds of arguments that are likely to have traction. Not that tenure is good for professors, or things like academic freedom that a minority of people care about. But tenure is good for students.

The flip side of that is that we can’t profess that tenure helps students and then denigrate or simply neglect teaching. Nor can we go along with “I won’t grade you too hard as long as you don’t demand too much.” Nor is this position compatible with allowing the system to continue to survive on contingent labor.

I’m still working out what the ethical thing to do is as someone who is (as we all are, in one way or another) caught up in this system. One thing I’m pretty sure about, though: appealing to faculty self-interest is not a winning strategy for gaining public support.

dear andrew perrin: i was wrong and you were right on the obesity and mortality correlation

A while back, Andrew and I got into an online discussion about the obesity/mortality correlation. He said it was true, I was a skeptic because I had read a number of studies that said otherwise. Also, the negative consequences of obesity can be mitigated via medical intervention. E.g., you may develop diabetes, but you can get treatment so you won’t die.

The other day, I wanted to follow up on this issue and it turns out that the biomedical community has come up with a more definitive answer. Using standard definitions of obesity (BMI) and mortality, Katherine Flegal, Broan Kit, Heather Orpana, and Barry I. Graubard conducted a meta-analysis of 97 articles that used similar measures of obesity and mortality. Roughly speaking, many studies report a positive effect, many report no effect, and some even report a negative effect. When you add them all together, you get a correlation between high obesity and mortality, but it is not true at ranges closer to non-overweight BMI. From the abstract of Association of All-Cause Mortality With Overweight and Obesity Using Standard Body Mass Index Categories: A Systematic Review and Meta-analysis, published in the 2013 Journal of the American Medical Association:

Conclusions and Relevance Relative to normal weight, both obesity (all grades) and grades 2 and 3 obesity were associated with significantly higher all-cause mortality. Grade 1 obesity overall was not associated with higher mortality, and overweight was associated with significantly lower all-cause mortality. The use of predefined standard BMI groupings can facilitate between-study comparisons.

In other words, high obesity is definitely correlated with mortality (Andrew’s claim). Mild obesity and “overweight” are correlated with less mortality (a weaker version of my claim). The article does not settle the issue of causation. It can be very likely that less healthy people gain weight. E.g., people with low mobility may not exercise or take up bad diets. Or people who are very skinny may be ill as well. Still, I am changing my mind on the basic facts – high levels of obesity increase mortality.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

jerry davis on the importance of management research

Harvard Business Review has run a version of Jerry Davis’ essay on the merits of modern management research. A few clips:

Is management research a folly? If not, whose interests does it serve? And whose interests should it serve?

The questions of good for what and good for whom are worth revisiting. There is reason to worry that the reward system in our field, particularly in the publication process, is misaligned with the goals of good science.

There can be little doubt that a lot of activity goes into management research: according to the Web of Knowledge, over 8,000 articles are published every year in the 170+ journals in the field of “Management,” adding more and more new rooms. But how do we evaluate this research? How do we know what a contribution is or how individual articles add up? In some sciences, progress can be measured by finding answers to questions, not merely reporting significant effects. In many social sciences, however, including organization studies, progress is harder to judge, and the kinds of questions we ask may not yield firm answers (e.g., do nice guys finish last?). Instead we seek to measure the contribution of research by its impact.

And:

Management of humans by other humans may be increasingly anachronistic. If managers are not our primary constituency, then who is? Perhaps it is each other. But this might lead us back into the Winchester Mystery House, where novelty rules. Alternatively, if our ultimate constituency is the broader public that is meant to benefit from the activities of business, then this suggests a different set of standards for evaluation.

Businesses and governments are making decisions now that will shape the life chances of workers, consumers, and citizens for decades to come. If we want to shape those decisions for public benefit, on the basis of rigorous research, we need to make sure we know the constituency that research is serving.

Required reading.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

can powerful, elite-led organizations lessen inequality?

Hi all, I’m Ellen Berrey. I’ll be guest blogging over the next few weeks about inequality, culture, race, organizations, law, and multi-case ethnography. Thanks for the invite, Katherine, and the warm welcomes! Here’s what I’m all about: I’m an assistant professor of sociology at the University at Buffalo-SUNY and an affiliated scholar of the American Bar Foundation. I received my PhD from Northwestern in 2008. This fall, I jet off from the Midwest to join the faculty of the University of Denver (well, I’m actually going to drive the fading 2003 Toyota I inherited from my mom).  

As a critical cultural sociologist, I study organizational, political, and legal efforts to address inequality. My new book, The Enigma of Diversity: The Language of Race and the Limits of Racial Justice (University of Chicago Press), is officially out next Monday (yay!). I’ll dive into that in future posts, for sure. I’m writing up another book on employment discrimination litigation with Robert Nelson and Laura Beth Nielsen, Rights on Trial: Employment Civil Rights in Work and in Court.  These and my articles and other projects explore organizational symbolic politics, affirmative action in college admissions (also here and here), affirmative action activism (and here), corporate diversity management, fairness in discrimination litigation, discrimination law and inequality (and here), gentrification politics, and benefit corporations.

I’ll kick off today with some thoughts about a theme that I’ve been exploring for many years:

How can powerful, elite-led organizations advance broad progressive causes like social justice or environmental protection? I’m not just referring to self-identified activists but also corporations, universities, community agencies, foundations, churches, and the like. Various arms of the state, too, are supposed to forward social causes by, say, ending discrimination at work or alleviating poverty. To what extent can organizational decision-makers create positive social change through discrete initiatives and policies—or do they mostly just create the appearance of effective action? Time and again, perhaps inevitably, top-down efforts to address social problems end up creating new problems for those they supposedly serve.

To the point: Have you come across great research that examines how organizations can bring about greater equality and engages organizational theory?

I think this topic is especially important for those of us who study organizations and inequality. We typically focus on the harms that organizations cause. We know, for example, that employers perpetuate racial, class, and gender hierarchies within their own ranks through their hiring and promotion strategies. I believe we could move the field forward if we also could point to effective, even inspiring ways in which organizations mitigate inequities. I have in mind here research that goes beyond applied evaluations and that resists the Polly Anna-ish temptation to sing the praises of corporations. Critical research sometimes asks these questions, but it often seems to primarily look for (and find) wrongdoing. Simplistically, I think of this imperative in terms of looking, at once, at the good and bad of what organizations are achieving. Alexandra Kalev, Frank Dobbin, and Erin Kelly’s much-cited American Sociological Review article on diversity management programs is one exemplar. There is room for other approaches, as well, including those that foreground power and meaning making. Together with the relational turn in the study of organizational inequality, this is a promising frontier to explore.

More soon. Looking forward to the conversation.