irb misconceptions
Last Fri., I attended a talk by Sarah Babb of Boston College. In her talk, titled “Beyond the Horror Stories: Non-Experimental Social Researchers’ Encounters with Institutional Review Boards (IRB),” Babb revealed findings that included misconceptions about federal guidelines for human subjects. Contrary to what some IRB review boards demand from principal investigators (PIs) undertaking qualitative research, the federal guidelines do not require:
- signed consent from a low risk population
- an institutional research permission slip
To repeat, the above two are “not in federal regulations at all.”
Babb noted that at larger institutions, IRB boards often involve nonprofessionals – that is, those who don’t have appropriate professional expertise – in the decision-making processes about proposals. Moreover, qualitative research don’t fit well into the one-size-fits-all medical template often used to vet research proposals. Compounding these challenges is the lack of accountability in terms of IRB review boards’ responsibilities to PIs. Only 20% of IRBs that Babb examined had an appeals procedure that would allow PIs to contest decisions.
Not surprisingly, this talk evoked spirited discussion of the myriad problems encountered by researchers going through the IRB process at their institutions, as well as the unintended consequences of a review process ostensibly intended to protect human subjects. The audience noted the following unintended and undesired consequences: (1) normalized deviance,* (2) chilling effect upon the types of research undertaken, and (3) mission creep in which IRB review boards critique the suitability or worth of the research design, rather than evaluating risk to human subjects. In particular, senior researchers worried that tenure-track faculty and graduate students face great uncertainty about whether their project proposals will successfully navigate the IRB process in a timely fashion.
Audience members asked whether the sociologists’ professional association, the American Sociological Association (ASA), had taken an official position on IRB guidelines. None present were aware of any such activities (if you know of anything brewing from this or other associations, do write them in the comments). Attendees noted that because a tenured faculty member may be more able to surmount IRB issues on his/her own (or not need to go through the IRB process because of the type of research conducted), fashioning IRB standards that are more appropriate for a wider variety of research methods is a collective action problem.
I opined that these identified problems need to be considered a commons issue. Those with more power should consider it a professional responsibility to help budding researchers – undergraduate students, graduate students, junior faculty – go through an IRB process that is appropriate to their research methods and questions, especially if researchers hope to have future generations of audiences and colleagues. Unfortunately, dark humor may not be sufficient to get the point across – when a psychology colleague sent his IRB board a proposal to reproduce the Stanley Milgram experiment on April Fool’s Day, an IRB staffer called to inquire if the proposal was serious.
* One of my past posts discussing the IRB draws a steady stream of traffic from those searching for the answer to one of the quiz questions on the online Collaborative Institutional Training Initiative (CITI), a certification program mandatory for researchers and students at some institiutions.
the scandal triangle
Brendan Nyhan has a nice post on the sociology of scandal. He summarizes his research on presidential scandal in this way:
My research suggests that the structural conditions are strongly favorable for a major media scandal to emerge. First, I found that new scandals are likely to emerge when the president is unpopular among opposition party identifiers. Obama’s approval ratings are quite low among Republicans (10-18% in recent Gallup surveys), which creates pressure on GOP leaders to pursue scandal allegations as well as audience demand for scandal coverage. Along those lines, John Boehner is reportedly “obsessed” with Benghazi and working closely with Darrell Issa, the House committee chair leading the investigation. You can expect even stronger pressure from the GOP base to pursue the IRS investigations given the explosive nature of the allegations and the way that they reinforce previous suspicions about Obama politicizing the federal government.
In addition, I found that media scandals are less likely to emerge as pressure from other news stories increases. Now that the Boston Marathon bombings have faded from the headlines, there are few major stories in the news, especially with gun control and immigration legislation stalled in Congress. The press is therefore likely to devote more resources and airtime/print to covering the IRS and Benghazi stories than they would in a more cluttered news environment.
I’d also add that “events” have properties. It is easier to scandalize, say, the IRS investigation issue because it is simple. In contrast, the issue of whether the attack in Libya should have been labeled terrorism is probably to esoteric for most folks. If you buy that argument, you get a nice story about the “scandal triangle.” The likelihood of scandal increases when partisan opposition, bored media, and clearly norm-broaching events come together.
Adverts: From Black Power/Grad Skool Rulz
coaches are the highest paid state employees
Deadspin provides this illustration of the highest paid state employees. It says something when a public institutions gives more support for collegiate sports than other functions. That’s why I advocate a separation of sport and school.
Adverts: From Black Power/Grad Skool Rulz
intrinsic motivation and the org
A guest post by Tim Sullivan, author of The Org
One of the most widely misinterpreted (or, in an interpretation less generous to Ray and me, let’s just say “confusing”) chapters of The Org is our third, which focuses on the motivation of Methodist ministers. We use research done by Chris Parsons and Jay Hartzell together with David Yermack on the Methodists of Oklahoma as a way of talking about incentives and organizational design. Some have interpreted the chapter to mean that we think Methodists should be paid more or less, or that we’re somehow finding fault with the denomination; others have been offended that we’ve compared the Methodists to Proctor & Gamble, the multinational consumer goods company; others have been shocked (shocked!) that we “accuse” some ministers of sheep-stealing.
Let me try a slightly different tack, placing the research in the context of intrinsic motivation.
There’s a wide-ranging literature in business about the importance of having a self-motivated workforce, from the popular and serious (Dan Pink’s Drive) to the popular and slightly less serious (articles with headlines like “Let Your Employees Decide What to Do”). And, of course, Ray and I would agree that having employees whose own motivations fit well with what the business is trying to accomplish is really important — vitally important, in fact. Actually, as a result of having written The Org, I’m even more convinced about the importance of “soft” skills and qualities like employee fit, talent management, and leadership.
But by itself, intrinsic motivation is not enough. Most businesses have to use incentives to help direct even the most well-intentioned, appropriately skilled employees who really believe in the business and want to do well by their employers and customers.
That’s where the research into the Methodists of Oklahoma comes in. No one could possibly argue that Methodist ministers don’t have intrinsic motivation a-plenty. They aren’t paid very well, they don’t have very high social status, they can have tremendously stressful working conditions, they have to manage a direct democracy, often with limited resources, and so on. They have no other reason to become ministers other than a strong avocation. We should all be so lucky in finding the right people to work with and for us.
But relying on their desire to do a good job is not enough. There are competing interests in the Methodist church, as there are in any organization, and as a result the definition of doing a “good job” can be murky. So the Methodists use some subtle incentives to encourage certain kinds of behaviors – even those that might not appeal to individual ministers.
Consider the issue of growth: the growth opportunities of local parishes may not exactly align with those of the statewide church, yet the responsibility for increasing both falls on pastors. No matter how intrinsically motivated they might be, they have to make choices.
Here’s the potential conflict. Local churches can grow by adding two different kinds of new parishioners: new converts or Methodists from other parishes. The former contributes to the growth of both the local parish and the statewide church; the latter does not. But the latter is, for some ministers, is far easier.
The bishop delegates the setting a pastor’s pay to a local Pastor-Parish Relations Committees. Again, this makes sense, since a local committee can closely observe a pastor and make sure he or she is doing well by the parish. But these committees tend to like it—for whatever reason—when their pastor steals sheep. Basically, Parsons and Hartzell found that PPRCs give pastors on average about $17 for converting someone new to Methodism but a whopping $35 for attracting a Methodist from a different parish.
Why would the central church allow the PPRC to reward sheep stealing? Because the bishop has another carrot: promotion. The bishop decides on who gets to move to a new, bigger church. By these lights, converting someone to Methodism averages as about a $200 gain, comparing the old salary to the new one; attracting a Methodist, only about $40.
This isn’t to say that these incentives and structures have been all carefully thought out and deliberately put into action. Far from it: it’s more likely that the Methodists in Oklahoma stumbled into this arrangement, but it suits their purposes and, more to the point, it helps direct the behavior of well-meaning, intrinsically motivated ministers who need a little directional nudge to help them figure out how to spend their time.
more tweets, more votes – cause or correlation?
A number of people have asked me a very important question about the More Tweets, More Votes paper. Do relative tweet rates merely correlate with elections or is there is a causal link?
The paper itself does not settle the issue. The purpose of the paper is merely to document this striking correlation. Given that qualification, let me explain the argument from both sides and my priors.
- Correlation: Twitter is a passive record of how excited people are. If a candidate somehow garners the attention of the public, they get excited and start talking about it, which translates into a higher twitter presence.
- Causal: The unusual attention that a candidate attracts in social media sways undecided or weakly committed voters. In a sense, highly active twitter users are the “opinion leaders” of modern society.
My prior: 75% correlation, 25% cause. How would tease out these arguments? For example, what variable could instrument the district level tweet counts? Interesting to find out.
Adverts: From Black Power/Grad Skool Rulz
restructuring in higher education, australian edition
Guy Tuchman has an article in Inside Higher Education that discusses big budget cuts in Australia:
Several processes are occurring simultaneously. Departments are being lumped together to form new constellations of schools and faculties. (In Australian academic parlance, schools are a bureaucratic unit composed of disciplines or programs. A faculty consists of a collection of schools.) But, my newly met colleagues insisted, restructuring is not about encouraging interdisciplinarity or intellectual cross-fertilization by increasing the administrative proximity of related fields of inquiry.
Rather, I was told, central administrators have been combining units as universities pare down the number of schools and faculties they harbor to extirpate unnecessary courses, eliminate “redundant” workers, and increase what the Aussies, like the British, call “casualization” and Americans term “contingent labor.” As in the United States, such practices are more likely to be applied to the arts, humanities and social sciences rather than to the STEM fields — science, technology, engineering, mathematics — which supposedly promise to raise needed income through contracts, grants, and inventions and to goose the region’s and the nation’s ability to succeed in international economic competition.
