orgtheory.net

Archive for the ‘research’ Category

the grant economy as tragedy of the commons

Grants, grants, grants. Every academic likes getting them. And every administrator wants more of them. In some fields, you can’t be an active researcher without them. In all fields, the search for grants grows ever more competitive.

PLOS recently published the results of a survey of the grant-writing process in the U.S. Drawing on a nonrandom sample of astronomers and social & personality psychologists, and asking about their applications to NASA (astronomers), NIH (psychologists), and NSF (both), they reported their researchers having applied for four grants in the past four years, on average, and received one. Proposals took an average of 116 hours to write.

While respondents did note some non-pecuniary benefits to grant-writing, comments suggested that many researchers simply gave up after repeated failures, judging it a poor use of their time. In the words of one respondent:

I applied for grants from the NSF in 2004, 2005, 2006 and 2007….Most of the reasons given for not funding were that funds were too tight that particular year and that I should reapply the next year since the proposal had merit…I finally just gave up.

The authors, who start with the certainly-debatable-but-not-totally-baseless assumption that funding is fairly random (see these cites), make some rough calculations to come to the conclusion that when the funding rate is at 20%, half of grant applicants will be forced to abandon their efforts after three failed attempts, with a rate of more like 2/3 for previously unfunded researchers.

Now, this is a limited study and there is plenty of room to argue about the data it uses and the assumptions it makes. But few would disagree with its conclusion: “20% funding rates impose a substantial opportunity cost on researchers by wasting a large fraction of the available research time for at least half of our scientists, reducing national scientific output.”

And 20% is the good scenario these days. The National Institute on Aging, one of NIH’s major funders of sociology research, recently announced paylines at the 7th (for under-$500k grants) and 4th (for over-$500k grants) percentiles. That’s incredibly competitive.

Yet at the same time, universities, desperate for revenues, are pushing harder and harder for researchers to bring in grants, with those sweet, sweet indirect costs. I know mine is. When I arrived at SUNY Albany seven years ago, I was happy that the university seemed to treat grants as a bonus—something nice to have, not something required by them. This was a contrast with at least one other place I interviewed.

But that’s less the case now. I won’t go into details here. But suffice it to say that the pressure on the department to bring in grants is steady and increasing. And while this may not be happening everywhere, it’s certainly common, particularly at cash-strapped public universities. It’s even worse, of course, in the soft-money parts of higher ed, like med schools.

The thing is, it’s simply not a viable strategy. Unless the pool of grant funding is massively increased at the federal level—a remote possibility—this is a zero-sum game. And so we’re left with what is basically a variation on the tragedy of the commons problem. Rational individuals (or individuals responding to their employers’ rational demands) will write more grant applications, since doing so still probably increases one’s chances of being funded. And if everyone else is working harder and harder to secure grant funding, maintaining constant effort will likely result in a decreasing payoff.

But what a collective waste of resources. The PLOS article cites research suggesting that university faculty spend an average of 10.7 (public universities) to 14.5 (private universities) hours per week on research during the semester. Even assuming grant-writers are finding somewhat more time for research, grant-writing still takes half a semester’s research time or so. Yes, there’s some intellectual benefit to the grant-writing process. But probably not so much to writing multiple unfunded grants.

My modest proposal is to cap the number of applications—either at the individual level, or the institutional level. Or perhaps establish a lottery for the right to submit in a given round. I can think of disadvantages to that—why should the best (or at least most fundable) researchers not be able to apply as often as they want? But the goal here isn’t to share the wealth—it’s to waste less of our collective time.

In the meanwhile, the ever-growing pressure for external funding in an environment of flat-at-best resources is itself throwing money out the window. At a time of intense pressure to do more with less, this is the biggest irony of all.

Written by epopp

March 16, 2015 at 12:30 pm

Posted in academia, research

questionable hypothesizing

Heads turn whenever accusations are made about academic impropriety, but it is especially provocative when one of the researchers involved in a study makes accusations about his/her own research. A forthcoming article in the Journal of Management Inquiry written by an anonymous senior scholar in organizational behavior does exactly this. The author, who remains anonymous to protect the identity of his/her coauthors, claims that research in organizational behavior routinely violates norms of scientific hypothesis testing.

I want to be clear: I never fudged data, but it did seem like I was fudging the framing of the work, by playing a little fast and loose with the rules of the game—as I thought I understood how the game should be played according to the rules of the scientific method. So, I must admit, it was not unusual for me to discover unforeseen results in the analysis phase, and I often would then create post hoc hypotheses in my mind to describe these unanticipated results. And yes, I would then write up the paper as if these hypotheses were truly a priori. In one way of thinking, the theory espoused (about the proper way of doing research) became the theory-in-practice (about how organizational research actually gets done).

I’m certain that some people reading this will say, “Big deal, people reformulate hypotheses all the time as they figure out what their analysis is telling them.”  The author recognizes this is the case and, I believe, relates his/her experience as a warning of how the field’s standards for writing are evolving in detrimental ways. For example, the author says, “there is a commonly understood structure or “script” for an article, and authors ask for trouble when they violate this structure. If you violate the script, you give the impression that you are inexperienced. Sometimes, even the editor requires an author to create false framing.” Sadly, this is true.

All too often reviewers feel that it is their role to tell the authors of a paper how to structure hypotheses, rewrite hypotheses, and explain analysis results. Some reviewers, especially inexperienced ones, may do this because they feel that they are doing the author(s) a favor – they’re helping make the paper cleaner and understandable. But the unintended consequence of this highly structured way of writing and presenting results is that it forces authors into a form of mild academic dishonesty in which they do not allow the exploratory part of the analytical process to be transparent.

Some journals have a much stronger ethos about hypothesis testing than others. AJS is looser and allows authors more freedom in this regard. But some social psych journals (like JPSP) have become extremely rigid in wanting to see hypotheses stated a priori and then tested systematically.  I would love to see more journals encourage variability in reporting of results and allow for the possibility that many of our results were, in fact, unexpected. I would love it if more editors chastised reviewers who want to force authors into a highly formulaic style of hypothesizing and writing results. It simply doesn’t reflect how most of us do research.

Perhaps the anonymous author’s tale will ignite a much needed discussion about how we write about social scientific analysis.

Written by brayden king

February 8, 2015 at 3:57 pm

party in the street: social identities and policy continuity

One of the issues that we draw attention to in Party in the Street is that there was a great deal of continuity in war policy between the Bush and Obama administrations. This is an example of a broader theme in American government: domestic disputes are not brought into foreign policy. The phrase for this is “politics ends at the water’s edge.”

The water’s edge idea has important implications for social movements, especially progressive movements that are often participating in anti-war activism. Normally, we think of movements responding to some sort of stark contrast in policy and they expect different political actors to have distinct views on policy. For example, it is pretty safe to say that Democrats and Republican leaders promote very different abortion policies.

In contrast, the “water’s edge” theory suggests that there will be a fair amount of continuity between administrations in terms of foreign policy. It doesn’t mean total similarity, but a great deal of overlap. For example, the Iraq withdrawal was initially negotiated by the Bush administration and then carried out by Obama’s administration. Similarly, both the Bush and Obama administrations, at various times, sought to extend US involvement in Iraq. Did both administrations have identical policy? Definitely not, but there is a lot of continuity and overlap.

If you believe that movements closely follow policy, then the overall path of the antiwar movement might seem puzzling. When Bush surged, the movement began its decline. As Obama sought extensions in Iraq, there was little protest. Antiwar activists did not focus on the main instrument of withdrawal, the Status of Forces Agreement, initiated by Bush. The War on Terror involved over 100,000 troops “on the ground” from 2003 till about 2010.

We argue in Party in the Street that the overall growth and decline of the antiwar movement can be better explained by the tension of activism and partisanship instead of policy shifts. Early on, the antiwar movement’s identity did not conflict with its ally, the Democratic Party. So the movement could draw partisans and grow during the early stages of the Iraq War. As elections changed the landscape, partisanship asserted itself and the movement ebbed. And that is how we get a declining movement as the US intervention in Iraq is sustained, then incrementally reduced during a multi-year withdrawal phase and the vastly expanded in Afghanistan.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street!! 

Written by fabiorojas

February 6, 2015 at 12:01 am

asian american privilege? a skeptical, but nuanced, view, and a call for more research – a guest post by raj ghoshal and diana pan

Raj Andrew Ghoshal is an assistant professor of sociology at Goucher College and Yung-yi Diana Pan is an assistant professor of sociology at Brooklyn College. This guest post is a discussion of Asian Americans and their status in American society.

As a guest post last month noted, Asian Americans enjoy higher average incomes than whites in the United States. We were critical of much in that post, but believe it raises an under-examined question: Where do Asian Americans stand in the US racial system? In this post, we argue that claims of Asian American privilege are premature, and that Asian Americans’ standing raises interesting questions about the nature of race systems.

We distinguish two dimensions of racial stratification: (1) a more formal, mainly economic hierarchy, and (2) a system of social inclusion/exclusion. This is a line of argument developed by various scholars under different names, and in some ways parallels claims that racial sterotypes concern both warmth and competence. We see Asian Americans as still behind in the more informal system of inclusion/exclusion, while close (but not equal) to whites in the formal hierarchy. Here’s why.

Read the rest of this entry »

Written by fabiorojas

February 4, 2015 at 12:01 am

christakis’ query

Last year, Nicholas Christakis argued that the social sciences were stuck. Rather that fully embrace the massive tidal wave of theory and data from the biological and physical sciences, the social sciences are content to just redo the same analysis over and over. Christakis’ used the example of racial bias. How many social scientists would be truly shocked to find that people have racial biases? If we already know that (and we do, by the way), then why not move on to new problems?

Christakis’ was recently covered in the media for his views and for attending a conference that tries to push this idea. To further promote this view, I would like to introduce Christakis’ Query, which every researcher should ask:

Think about the major question that you are working on and what you think the answer is. Estimate the confidence in your answer. If you already know the answer with more than 50% confidence, then why are you working on it? Why not move on?

Try it out.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($1!!!!)/From Black Power/Party in the Street!! 

Written by fabiorojas

January 27, 2015 at 12:02 am

defending computational ethnography

Earlier this week, I suggested a lot is to be gained by using computational techniques to measure and analyze qualitative materials, such as ethnographic field notes. The intuition is simple. Qualitative research uses, or produces, a lot of text. Normally, we have to rely on the judgment of the researcher. But now, we have tools that can help us measure and sort the materials, so that we have a firmer basis on which to make claims about what our research does and does not say.

The comments raised a few issues. For example, Neal Caren wrote:

 This is like saying that you want your driverless cars to work for Uber while you are sleeping. While it sounds possible, as currently configured neither ethnographic practices nor quantitative text analysis are up to the task.
This is puzzling. No one made this claim. If people believe that computers will do qualitative work by collecting data or developing hypotheses and research strategies, then they are mistaken. I never said that nor did I imply it. Instead, what I did suggest is that computer scientists are making progress on detecting meaning and content and are doing so in ways that would help research map out or measure text. And with any method, the researcher is responsible for providing definitions, defining the unit of analysis and so forth. Just as we don’t expect regression models to work “while you are sleeping,” we don’t expect automated topic models or other techniques to work without a great level of guidance from people. It’s just a tool, not a magic box.
Another comment was meant as a criticism, but actually supports my point. For example, J wrote:
This assumes that field notes are static and once written, go unchanged. But this is not the consensus among ethnographers, as I understand the field. Jonathan van Maanen, for example, says that field notes are meant to be written and re-written constantly, well into the writing stage. And so if this is the case, then an ethnographer can, implicitly or intentionallly, stack the deck (or, in this case, the data) in their favor during rewrites. What is “typical” can be manipulated, even under the guise of computational methods.
Exactly. If we suspect that field notes and memos are changing after each version, we can actually test that hypothesis. What words appear (or co-appear) in each version? Do word combinations with different sentiments or meanings change in each version? I think it would be extremely illuminating to see what each version of an ethnographer’s notes keeps or discards. Normally, this is impossible to observe and, when reported (which is rare), hard to measure. Now, we actually have some tools.
Will computational ethnography be easy or simple? No. But instead of pretending that qualitative research is buried in a sacred and impenetrable fog of meaning, we can actually apply the tools that are now becoming routine in other areas for studying masses of text. It’s a great frontier to be working in. More sociologists should look into it.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($1!!!!)/From Black Power/Party in the Street!! 

Written by fabiorojas

January 23, 2015 at 12:01 am

party in the street: the main idea

For the last eleven years, my friend Michael Heaney and I have conducted a longitudinal study of the American antiwar movement. Starting at the 2004 Republican National Convention protests in New York City, we have been interviewing activists, going to their meetings, and observing their direct actions in order to understand the genesis and evolution of social movements.  We’ve produced a detailed account of our research in a new book called Party in the Street: The Antiwar Movement and the Democratic Party after 9/11. If the production process goes as planned, it should be available in February or early March.

In our book, we focused on how the antiwar movement is shaped by its larger political environment. The argument is that the fortunes of the Democratic party affect the antiwar movement’s mobilization. The peak of the movement occured when the Democratic party did not control either the White House or Congress. The movement demobilized as Democrats gained more control over the Federal government.

We argue that the the demobilization reflects two political identities that are sometimes in tension: the partisan and the activist. When partisan and activist goals converge, the movement grows as it draws in sympathetic partisans. If activism and partisanship demand different things, partisan identities might trump the goals of activist, leading to a decline of the movement. We track these shifting motivations and identities during the Bush and Obama administrations using data from over 10,000 surveys of street protestors, in depth interviews with activists, elected leaders, and rank and file demonstrators, content analysis of political speeches, legislative analysis, and ethnographic observations.

If you are interested in social movements, political parties and social change, please check it out. Over the next month and a half, I will write posts about the writing of the book and the arguments that are offered.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($1!!!!)/From Black Power/Party in the Street!!

Written by fabiorojas

January 6, 2015 at 12:01 am

Follow

Get every new post delivered to your Inbox.

Join 3,088 other followers