Archive for the ‘fabio’ Category
A recent report on software quality:
As projects surpass one million lines of code, there’s a direct correlation between size and quality for proprietary projects, and an inverse correlation for open source projects. Proprietary code analyzed had an average defect density of .98 for projects between 500,000 – 1,000,000 lines of code. For projects with more than one million lines of code, defect density decreased to .66, which suggests that proprietary projects generally experience an increase in software quality as they exceed that size.
In other words, open source works for small projects, but proprietary big projects have better quality. Why? A few hypotheses:
- Incentives – maybe the joy of fixing code is simply washed out for big projects. You would only bother if there was a pay off.
- Teams & management – maybe large projects simply require large teams of dedicated people, which is hard to do in the open source world.
- Selection – maybe for-profits only will support a project if it is easy to maintain and thus reduce costs.
Brendan Nyhan has a nice post on the sociology of scandal. He summarizes his research on presidential scandal in this way:
My research suggests that the structural conditions are strongly favorable for a major media scandal to emerge. First, I found that new scandals are likely to emerge when the president is unpopular among opposition party identifiers. Obama’s approval ratings are quite low among Republicans (10-18% in recent Gallup surveys), which creates pressure on GOP leaders to pursue scandal allegations as well as audience demand for scandal coverage. Along those lines, John Boehner is reportedly “obsessed” with Benghazi and working closely with Darrell Issa, the House committee chair leading the investigation. You can expect even stronger pressure from the GOP base to pursue the IRS investigations given the explosive nature of the allegations and the way that they reinforce previous suspicions about Obama politicizing the federal government.
In addition, I found that media scandals are less likely to emerge as pressure from other news stories increases. Now that the Boston Marathon bombings have faded from the headlines, there are few major stories in the news, especially with gun control and immigration legislation stalled in Congress. The press is therefore likely to devote more resources and airtime/print to covering the IRS and Benghazi stories than they would in a more cluttered news environment.
I’d also add that “events” have properties. It is easier to scandalize, say, the IRS investigation issue because it is simple. In contrast, the issue of whether the attack in Libya should have been labeled terrorism is probably to esoteric for most folks. If you buy that argument, you get a nice story about the “scandal triangle.” The likelihood of scandal increases when partisan opposition, bored media, and clearly norm-broaching events come together.
A number of people have asked me a very important question about the More Tweets, More Votes paper. Do relative tweet rates merely correlate with elections or is there is a causal link?
The paper itself does not settle the issue. The purpose of the paper is merely to document this striking correlation. Given that qualification, let me explain the argument from both sides and my priors.
- Correlation: Twitter is a passive record of how excited people are. If a candidate somehow garners the attention of the public, they get excited and start talking about it, which translates into a higher twitter presence.
- Causal: The unusual attention that a candidate attracts in social media sways undecided or weakly committed voters. In a sense, highly active twitter users are the “opinion leaders” of modern society.
My prior: 75% correlation, 25% cause. How would tease out these arguments? For example, what variable could instrument the district level tweet counts? Interesting to find out.
Several processes are occurring simultaneously. Departments are being lumped together to form new constellations of schools and faculties. (In Australian academic parlance, schools are a bureaucratic unit composed of disciplines or programs. A faculty consists of a collection of schools.) But, my newly met colleagues insisted, restructuring is not about encouraging interdisciplinarity or intellectual cross-fertilization by increasing the administrative proximity of related fields of inquiry.
Rather, I was told, central administrators have been combining units as universities pare down the number of schools and faculties they harbor to extirpate unnecessary courses, eliminate “redundant” workers, and increase what the Aussies, like the British, call “casualization” and Americans term “contingent labor.” As in the United States, such practices are more likely to be applied to the arts, humanities and social sciences rather than to the STEM fields — science, technology, engineering, mathematics — which supposedly promise to raise needed income through contracts, grants, and inventions and to goose the region’s and the nation’s ability to succeed in international economic competition.
I occasionally teach a course aimed at business undergraduates. It’s a work/occupations/orgs course that uses various economic examples to discuss sociological ideas. The issue for me is that I often get torched in the evaluations. In my other classes, my evaluations range from the department average to very high. But hitting the department average is real accomplishment for this course. I’ve heard the same from some other instructors in the department. They do well with sociology students, but the identical course will get much lower scores when it is taught to business students.
So I ask my brothers and sisters in management: What would you advise the instructor of business students? In the past, I’ve added discussion, taken it away, added/subtracted readings, added/taken away group projects, provided my slides online, etc. How else can I experiment with this course?
When I visited Millsaps College a few weeks ago, I got into a discussion about international relations theory with my host, political scientist Michael Reinhard. I asked him why we (social scientists) needed to study famous political leaders, like Julius Caesar or Winston Churchill. His argument was intriguing. He said that highly successful social actors have often spent a lot of time understanding their social world. They are good at what they do – international relations in this case – because, at the very least, they have an intuition about the world that is important and correct. Some, like Churchill, will even explain their views to others. In other words, political scientists should study great leaders because great leaders actually understand power fairly well.
In sociology, we have no such argument, but it is worth thinking about. We are resistant to great leader stories and for good reason. Great man stories often devolve into hero worship, or they rely on “Whig” history. But that doesn’t mean Great people scholarship is not without use. For example, what did Steve Jobs understand about markets that management scholars should learn? Or, a more sociological example, what does a great religious leader understand about religion that sociologists of religion should know? Taking a turn from Bourdieu, we could look at any social field, identify the “masters,” and then use them as research sites where we can understand how the field is put together.
Last week, we had a discussion about academia and social mobility. Is it the case that low SES individuals are well served by a career in academia? My response is no. Graduate education is highly uncertain. Even if you get the degree there’s a good chance that you might be adjuncting. You might have to get work outside of academia that does not require doctoral education. I suggested that if we are really concerned about inequality, we’d suggest that people more seriously consider career paths that have high rewards and low risk, like engineering or health.
Then, Krippendorf wrote a comment that made me seriously question my claim. I quote the entire comment:
For Hispanic men, Hispanic women, and African American women, the estimated lifetime earnings of a PhD are greater than the estimated lifetime earnings of professional degree holder. The much-touted professional-to-PhD earnings drop is limited to whites, Asian Americans, and African American men. See here: http://www.census.gov/prod/2011pubs/acs-14.pdf, Table A2
If — and, taking Rory and others’ points, it’s a big if — the goal is to increase the earnings potential of students of color, using group averages as your sole predictor, it still doesn’t make sense to discourage all students of color from getting a PhD.
That being said, I completely agree that the solution isn’t admitting additional PhD students. (And, at my university, students of color are not “add-ons:” they count against the department’s allocation of both slots and funding packages, just like any other student.)
I agree with much in this comment. Group averages are not to be used strictly in all cases. I am also glad that Krippendorf and I agree that we shouldn’t be expanding graduate enrollments. But the core of Krippendorf’s comment made me think: is it really true that black MDs make *less* than black PhDs? If you look at the linked census report,* that is the case (see Table 2-C, for example). So what gives?
I suspect it has to do with the definition of “professional degree.” My hypothesis is that Blacks and Hispanics PhD make more than professionals because Blacks and Hispanics are less likely to be in high paying professions (e.g., MDs) and more likely to be in low paid professions (e.g.,social work a profession). The report does not list what counts as a “profession,” so it’s hard to say.
There is circumstantial evidence for my interpretation. For example, a 2009 article in Health Affairs estimates physician income by ethnic group. Not surprisingly, even Black and Latino MD’s make a bit more than the average for all professors. For example, the *average* Black male family practitioner makes about $159k a year. Hispanics actually earn *more* than their White counterparts on the average. We can also look at engineering. Not much research, but one study by NACME shows that engineering salaries for Black bachelor degree holders in their 30s is about $73k – which is a little above the average for all professor ranks combined in sociology. If you look at life time earnings, even Black engineering BA holders do better than Black PhD because they don’t have to spend a decade getting the degree, paying extra tuition, and loading up on debt.
Bottom line: There’s something fishy in that Census report and other evidence shows professions, at least STEM/health, are a better path for mobility for minorities.
* One of the authors is a former student, so I claim credit for all excellence in the report.
The Chronicle of Higher Education features a study of valedictorians and finds that class background affects where they apply to college:
Poorer students remain underrepresented at America’s top colleges, research has shown. And their academic preparation isn’t the only reason, according to Radford’s study of valedictorians, who should be considered well-prepared.
“Less-affluent valedictorians were less likely to know someone who had enrolled in a most selective institution and thus had a harder time envisioning their own attendance,” Radford wrote in a summary of her research.
The theme of the research association’s meeting this year was “Education and Poverty.” And Radford was among many who presented research on class inequity in higher education, which academics say remains deeply problematic at most colleges. Her study comes at a time of increased focus on how, despite plenty of outreach efforts, much of the talent at low-income high schools isn’t getting recruited to top colleges.
Radford worked with data from the High School Valedictorian Project, a survey of 900 class valedictorians who graduated from public high schools between 2003 and 2006. She also drew from 55 in-depth interviews with the students. The University of Chicago Press soon will publish a book by Radford on her findings.
This is probably one of the key findings of recent stratificiation research. Class doesn’t affect life course only through material resources, but by changing the habitus.
In age of climate denialism and other chicanery, it’s easy to be a science pessimist. But when I stand back, I become a little more confident about things. Science, as an institution, has not buckled under pressure. For example, I think about vaccine skeptics. Truly bad science that has lead to some deaths. However, science did not abandon vaccines and instead went in search of confirmatory evidence and found nil. This was before the retraction of the infamous article in Lancet.
People may sneer at the social sciences, but they hold up as well. Recently, a well known study in economics was found to be in error. People may laugh because it was an Excel error, but there’s a deeper point. There was data, it could be obtained, and it could be replicated. Fixing errors and looking for mistakes is the hallmark of science. In sociology, we often shy away from the mantle of science, but our recent treatment of the Regnerus paper makes me proud. My fellow sociologists obtained the data, analyzed it, and showed that the new data support the long standing finding of no differences between same sex and different sex parents in terms of childhood outcomes.
If you watch the news, the Coburns of the world claim the attention. But when you think about it, the science haters are really standing in the shadow of a much larger enterprise.
I am one of those people who thinks that we should not encourage people to enter the academic profession unless they are extremely committed to scholarship and they show exceptional promise. This advice often triggers a reaction that is summarized as: “You are evil! You want to exclude poor people/minorities/women/others from academia!”
My response: encouraging an expansion of graduate education does not address most aspects of inequality and might make it worse in many cases. For example, there is a large scale gap between whites and blacks in terms of education, income, and wealth. Sending people to graduate school will not address this gap. There are many reasons: lots of people don’t finish the degree; huge opportunity costs; low paid adjunct work after graduation; accumulation of burdensome of debt; and the tenure track pays modestly compared to other professionals with similar qualifications. These trends suppress mobility.
In contrast, there lots of other professions that are much more likely to lead to good income and mobility. If we want to genuinely shrink the income gap between people of color and whites, for example, we are much wiser to encourage engineering and health science careers. You’ll get the degree in a few years and almost immediately jump higher in the income distribution. Way, way, way easier than going for that anthropology PhD and hoping for a tenure track job 12 years later.
If we want to address inequality within academia (ie., increasing representation on the faculty), we should reserve our efforts for getting people through the PhD pipeline and into jobs. We shouldn’t cram more graduate students into the pipeline. We should actually ask the logical question: What can we do to ensure that students acquire the right skills in academia? How can we make sure that they develop the right networks, that lead to publication in the “right” journals, and thus lead to the “right” jobs?
Sadly, very little effort goes into this side of things. It’s easier to count minorities and women and yell, “not fair! we need more!” It’s much harder to confront tenured faculty (like myself), and say: “Why haven’t you co-authored with women (or minorities) so that they may have a shot at a good tenure track job?” Let’s put the brakes on enrolling more students into doctoral programs and take up the less glamorous, but more important task, of making sure that the ones in the system will actually have the best careers possible.
The Open Borders movement is based around a simple idea – in most cases, people should not be restricted in their movement across borders. This idea was featured this weekend in The Atlantic. The article presents the case and discusses the academics and writers who congregate at the Open Borders blog, which is run by Vipul Naik.
Michael Huemer, a philosopher, boils down the argument with the hypothetical story inspired by the “Starvin’ Marvin” South Park character:
[Marvin] is very hungry and is trying to travel to the marketplace to buy some food. Another person, Sam (Sam has a large number of nephews and nieces, so we’ll call him Uncle Sam), decides to stop Marvin from going to the marketplace using coercion. He goes down there with his M16 and blocks the road. As a result, Marvin can’t trade for food and, as a result, he starves. So then the question is, did Sam kill Marvin? Did he violate his rights? Almost anyone would say yes, Sam acted wrongly. In fact, if Marvin died as a result, then Sam killed him. It wouldn’t be that Sam failed to help Marvin. No, he actively intervened….This is analogous to the U.S. government’s immigration policy. There are people who want to trade in our marketplace, in this case the labor market, and the government effectively prevents them from doing that, through use of force.
I was also cited for discussing open borders strategy:
“Open borders will become a reality when the public stops believing that immigrants are a threat,” sociologist Fabio Rojas recently wrote, comparing the open borders movement to the gay rights movement. “Even if a pro-immigration referendum fails to pass, it will still serve the function of forcing the issue onto the public stage. These actions won’t change the minds of those strongly committed to anti-immigration policy. Instead, they will make immigration seem ‘normal’ to a later generation of people.”
Check it out.
When people read our More Tweets, More Votes paper, they often wonder – where is the “sentiment analysis?” In other words, why don’t we try to measure whether a tweet is positive or negative? Joe DiGrazia, the lead author, addressed this in a recent interview with techpresident.com:
DiGrazia said the researchers were “kind of surprised” that they saw a correlation without doing sentiment analysis of the Tweets. “We thought we were going to have to look at the sentiment,” he said. He speculated that one reason for the correlation could be a so-called Pollyanna Hypothesis, “that people are more likely to gravitate toward subjects that they are positive about and are more likely to talk about candidates that they support.”
The idea is simply this: the frequency of speech is often a relatively decent approximation of how imporant people think that topic is relative to salient alternatives. If people say “Obama” a little more often than the competition, then it’s not unreasonable to believe that he is more favored. And you don’t need content analysis to suss that out.
attention stratification researchers: we now have seven social classes, i repeat: we now have seven social classes
From the UK, a new survey, conducted by the BBC and six universities, asserts that there are now seven social classes in Britain. The Guardian has a humorous take, using example from UK sitcoms:
Elite: General Melchett from Blackadder Goes Fourth. Braying, bellowing, incompetent and utterly contemptuous of the lower orders, Melchett would naturally expect to find himself at the top of the pecking order.
Established middle class: Margot and Jerry Leadbetter from The Good Life. As the establishment pillars of comfortable and conservative 1970s suburban society, the couple existed in pointed contrast to their more free-thinking neighbours Tom and Barbara Good.
Technical middle class: David Brent from The Office. Despite his supposedly rock’n'roll past, Ricky Gervais’s fist-gnawingly embarrassing general manager was resolutely middle class.
New affluent workers: Miranda from Miranda. Miranda Hart herself may be established middle class, but the heroine of her eponymous sitcom sits comfortably in a slightly lower category.
Traditional working class: Jim Royle from The Royle Family. Could Ricky Tomlinson’s armchair-bound, TV-addicted patriarch be anything other than proudly working class? My arse!
Emergent service workers: Maurice Moss from the IT Crowd. Young, nerdish and living at home with his mum, Moss could fit the emergent service worker class but probably needs a little work to increase his social and cultural capital levels.
Precariat: Rab C Nesbitt. Gregor Fisher’s much-loved and enduring sitcom creation has assumed the status of folk hero despite his resolutely unglamorous life.
This study investigates whether the increasingly common trend of working long hours (“overwork”) perpetuates gender segregation in occupations. While overwork is an expected norm in many male-dominated occupations, women, especially mothers, are structurally less able to meet this expectation because their time is subject to family demands more than is men’s time. This study investigates whether the conflicting time demands of work and family increase attrition rates of mothers in male-dominated occupations, thereby reinforcing occupational segregation. Using longitudinal data drawn from the Survey of Income and Program Participation, I show that mothers are more likely to leave male-dominated occupations when they work 50 hours or more per week, but the same effect is not found for men or childless women. Results also show that overworking mothers are more likely to exit the labor force entirely, and this pattern is specific to male-dominated occupations. These findings demonstrate that the norm of overwork in male-dominated workplaces and the gender beliefs operating in the family combine to reinforce gender segregation of the labor market.
Apparently, yes. An article in Talking Points memo reports on a rare, but disturbing, aspect of our immigration laws. Hospitals may pay for undocumented immigrants to be moved to medical facilities in their original nation. They occasionally do this when people start in the emergency room, they stabilize, and then insurance does not pay for long term care:
Hundreds of immigrants who are in the U.S. illegally have taken similar journeys through a little-known removal system run not by the federal government trying to enforce laws but by hospitals seeking to curb high costs. A recent report compiled by immigrant advocacy groups made a rare attempt to determine how many people are sent home, concluding that at least 600 immigrants were removed over a five-year period, though there were likely many more.
To be sure, very uncommon, but it starkly points to a disturbing issue. Current law allows the state and other entities, hospitals in this case, to grossly violate one’s individual freedom if the aren’t a documented migrant. There’s a healthy debate to be had over the degree to which hospitals should provide care for the uninsured, but that doesn’t imply that somebody can be be shipped to another country because they are a non-citizen and it saves the hospital some money.
Unit of analysis: US House elections in 2010 and 2012. X-Axis: (# of tweets mentioning the GOP candidate)/(# of tweets mentioning either major party candidate). Y-axis: GOP margin of victory.
I have a new working paper with Joe DiGrazia*, Karissa McKelvey and Johan Bollen asking if social media data actually forecasts offline behavior. The abstract:
Is social media a valid indicator of political behavior? We answer this question using a random sample of 537,231,508 tweets from August 1 to November 1, 2010 and data from 406 competitive U.S. congressional elections provided by the Federal Election Commission. Our results show that the percentage of Republican-candidate name mentions correlates with the Republican vote margin in the subsequent election. This finding persists even when controlling for incumbency, district partisanship, media coverage of the race, time, and demographic variables such as the district’s racial and gender composition. With over 500 million active users in 2012, Twitter now represents a new frontier for the study of human behavior. This research provides a framework for incorporating this emerging medium into the computational social science toolkit.
The working paper (short!) is here. I’d appreciate your comments.
* Yes, he’ll be in the market in the Fall.
In this last post, I’ll discuss why I fundamentally disagree with the argument presented in Reinventing Evidence. There are two reasons. First, I agree with Andrew Perrin that Biernacki wants us to embrace a textual holism. One of Biernacki’s major arguments is that by isolating a single word, or passage, we are losing the entire meaning of the text. Thus, interpretation is the only valid approach to text. Coding and quantification is invalid. Perrin points out that lots of things be isolated. For example, if I see the n-word, I can say that, on the average, the text is employing racist language.
Second, Biernacki does not seem to consider cultural competence. In other words, human beings are creatures that can often reliably capture the meaning of utterances made by other humans from the same cultural group. Of course, I am talking about things like every day speech or short and simple writings like newspaper articles. More complex texts, like novels, will have networks or dense layering of meaning that go beyond a human’s native capacity for communication. These probably could be coded, but it would require intense training and an elaborate theory of text, which sadly we don’t have in sociology. But my major point remains. There’s a lot of fairly simple text that can be coded. If you believe that people can accurately convey the meaning of a text or label some aspect of it because they are “native speakers” of the culture, then coding is a valid thing to do. To believe otherwise, is to assume a world of solipsistic culture where every act of utterance requires a stupendous level of interpretation on the part of the audience.
So to wrap things up. I give credit to Biernacki for making us think hard about the quality of coding which is lacking. The fact that science is presented in ritual is fair, but doesn’t address whether a particular procedure produces valid measurement or inference. And I think that the view that texts are essentially uncodable is in error.
I am not sure what this accomplishes, but some journalists are trying to get further records from the University of Central Florida, where the editor of Social Science Research works. From the website of activist and author John Becker:
Despite the wide reach of the New Family Structures Study, much about the process by which it was peer-reviewed and published by the journal Social Science Research remains unknown. We know that the timetable was extraordinarily compressed — according to data from the University of Texas and SSR, Regnerus submitted his paper 20 days before the end of the data collection period and 23 days before the data file was delivered to the university. Sounds fishy, doesn’t it? And the entire process, including the paper’s initial submission, review, revision, and acceptance, took place within six weeks. But why? What are the reasons for moving so quickly? Did Regnerus just catch a lucky break, or is there more to the story? We already know that his funders had an anti-gay agenda and the study itself was plagued by troubling conflicts of interest; were the peer review and publication processes similarly compromised?
Last month, I filed a Freedom of Information Act request with the University of Central Florida, which houses Social Science Research, seeking public records relating to the peer review and publication of the New Family Structures Study. My goal is simply to discover the truth: whether everything was above board and best practices and ethical standards were followed, or whether something more sinister occurred. The documents I requested from UCF may help to answer these important questions.
A very learned commentary on the first edition of Dungeons and Dragons. For example, where “alignments” come from:
First of all, the paper explores crucial editorial mistakes in the production of the earliest version of original Dungeons & Dragons (OD&D). These are cases where some passages in OD&D are inconsistent with the remainder of the text in a way that hints at what early drafts of OD&D must have looked like. Previously, these have been curiosities to scholars of OD&D. Why does the elemental monster text refer to elemental controlling devices as “medallions, gems, stones or bracelets” instead of the names in the magical item list? Why does the languages passage refer to alignment languages as “divisional” languages? How did the percentage range for the “Ring of Delusion” end up broken? With the Dalluhn Manuscript in hand, we can find answers to all of these questions: each inconsistency points to the content of an earlier draft, a pre-publication system which is preserved in the Dalluhn Manuscript. For “divisional languages,” for example, we learn that “dvision” was the name for “alignment” in Dalluhn.
Required for nerds.
Article of interest from Rationality and Society:
- Mark Pingle and Tigran Melkonyan on “To believe or not believe…or not decide: A decision-theoretic model of agnosticism“
- David Pate on “Concealing to reveal: The informational role of Islamic dress“
- Douglas Savitski on “Is plea bargaining a rational choice? Plea bargaining as an engine of racial stratification and overcrowding in the United States prison system“
- Anthony Paik and Vernon Woodley on Symbols and investments as signals: Courtship behaviors in adolescent sexual relationships
- Louis Corriveau Game Theory and the Kula
Check it out.
Recently, sociologist Tressie McMillam Cottom wrote a column titled: “Does Blanket ‘Don’t Go to Graduate School!’ Advice Ignore Race and Reality?” It’s a nice article and worth reading. She makes the case that it is a mistake to tell people “Don’t Go to Graduate School” as a one-size-fits-all piece of advice. It’s the advice contained in chapter 1 of the Grad Skool Rulz ($3 - cheap!!). Specifically, she argues that graduate school represents an important avenue of mobility, especially for people of color who face disadvantages in the labor market.
A few responses from someone who advises a lot of students, in person and through my writings. Firsts, I don’t believe in advice that fits everyone. In the introduction of the book, I write: “I trust that you will be resourceful enough to adjust the advice for your own situation.” Human action is defined by circumstance, which is always variable and complex.
However, that doesn’t mean that you can’t offer advice aimed at typical students making typical choices. When it comes to graduate education, “don’t go to graduate school” is a sound starting point for discussion because graduate school is a bad choice for most people who have recently completed their B.A. Why?
- First, most students think the Ph.D. is needed for getting good jobs. Yes, there are some jobs that require a Ph.D. But most jobs do not require a Ph.D. degree, aside from university teaching or scientific research in the physical sciences or engineering.
- Second, most students do not understand that doctoral education is built around producing scholars. When people say, “I want to get a Ph.D.” I answer: “Do you want to be a college teacher?” The answer is usually no.
- Third, graduate education entails more risk than almost any other kind of professional education. As I’ve written before, the average Ph.D. doesn’t get the degree,while most students in other professional programs get the degree.
- Fourth, job prospects are very, very limited in many academic disciplines. Sociology’s market isn’t that bad, but it’s horrible in the humanities and the sciences where there is no external market.
- Fifth, academia entails great costs, even if you get the degree. You have limited geographical mobility. Partners may leave you. You make less money than other professionals with similar training.
In other words, academia is a neat place to work, but it is a very risky career and many people simply aren’t suited for it.
Cottom justifiably raises the issue of job market signalling. Perhaps the advice I just gave applies less to students of color because they’ve been stigmatized. They need to overcompensate through educational credentials in order to experience mobility.
Two responses: First, doctoral training is only one option. If one wants to signal intelligence on the labor market through credentials, you would probably be better served by other forms of education than the typical PhD program. For example, most engineering programs have master’s level work that leads to jobs. Other professions, such as teaching and business, also have short post-graduate courses of study that more directly lead to well compensated career tracks.
Second, there are serious opportunity costs. If one wants a job outside academia, it might better to work in a non-academic job to build experience. You earn a normal salary and get to know your field. While you spend 8 years on that Ph.D., you lose a lot of money and experience.
Let me conclude with an attempt to clarify my point. I am not trying to drive away people who are genuinely interested in the academic career. What I am doing is trying to sort between people who want to be scholars and those who seek training or credentials for employment outside higher education. The Ph.D. program is a very inefficient and risky way to pursue these goals.
To summarize: Richard Biernacki claims that coding textual materials (books, speech, etc) is tantamount to committing gross logical errors that mislead social scientists. Overall, I think this point is wrong but I think that Reinventing Evidence does a great service to qualitative research by showing how coding of texts might be critiqued and evaluated. In other words, ironically, by critiquing prior work on text coding, Biernacki draws our attention to the fact that qualitative research can be subjected to the same standards as quantitative research.
What do I mean? Well, a big problem with qualitative research is that it is very hard to verify and replicate. It is rare when ethographers go to the same field site, or informants are re-interviewed by others. A lot of the strength of quantitative research lies in the fact that other researchers can replicate prior results. For example, if I claim that party ID is correlated with gay marriage attitudes in the GSS, another researcher can download the same data and check the work. If they think the GSS made a mistake in collecting the data, a second survey can be conducted.
Biernacki, in trying to prove that coding qualitative data is pointless, follows a similar strategy by choosing a few articles of note and then he tries to reproduce the results. For example, he chooses Bearman and Stovel’s “Becoming a Nazi: A Model for Narrative Networks” which appeared in Poetics. The article creates a network out of ideas and themes mentioned from the memoir of a Nazi. Assuming that Biernacki reports his results correctly, he’s persuaded me that we need better standards for coding text. For example, he finds that Bearman and Stovel use an abbreviated version of the memoir – not the whole thing. Big problem. Another issue is how the network of text is interpreted. In traditional social network analysis, centrality is often thought to be a good measure of importance. Biernacki makes the reasonable argument that this assumption is flawed for texts. Very important ideas can become “background,” which means they are coded in a way that results in a low centrality score. This leads to substantive problems. For example, the Nazi mentions anti-semitism briefly, but in important ways. Qualitatively we know it is important, but the coding misses this issue.
Next week, I’ll get to my views on Biernacki’s attack on coding. But for now, I’ll give him credit for drawing my attention to these issues. The problems with the coding of the Nazi memoir point to me that there is more work to be done. We need to first start with a theory of text and then build techniques. If you want to use network analysis, you may have to take into consideration that standard network ideas may not be suitable. That will help us address problems like how to judge a text and the way we code data. That may not be the lesson Biernacki intended, but it’s a good one.