Archive for the ‘fabio’ Category
A few weeks ago, I suggested that one can use techniques from computer science to assess, measure, and analyze the field notes and interviews that one collects during field work. The reason is that computer scientists have made progress in writing algorithms that try to pick up the emotional tenor or meaning of texts. Not perfect by any means, but it would be a valuable tool that can be used to help qualitative researchers identify themes and patterns in the text.
In the last round, there were two comments that I want to address. First, Krippendorf wrote: “Why call it computational ethnography and not just text analysis?” Answer: There are two existing modes of analyzing text and techniques like sentiment analysis and topic modelling new things in new ways. Allow me to explain:
- The traditional way of reading qualitative texts is simply for the researcher to read the texts and develop a grounded understanding of the meaning that the text represents. This is the standard mode among historians, most anthropologists, and some sociologists. Richard Biernacki in Reinventing Evidence in Social Inquiry argued that is the only valid mode of qualitative analysis.
- The other major way to deal with qualitative materials is to conduct a two step operation of having people code the data (using key words or other instructions) and then performing an inter coder reliability analysis (i.e., assign codes to texts and compute Krippendorf alpha’s).
So what is new? Techniques like topic models or sentiment analysis do not use people to code data. After you train the algorithms, it is all automated. This has advantages – speed, reproducibility, and so forth – for large data. Another novel aspect is that these algorithms are usually built with some sort of model of language in mind that gives you insight into how the text was coded. For example, the Stanford NSL package essentially breaks down sentences by grammar and then estimates the distribution of words with specific sentiment. Thus, there is an explanation for every output. In contrast, I can’t reproduce even my own codes over time. Give me a set of text next week, and it will be coded a little different.
Second, a number of commenters were concerned about the open ended nature of notes, the volume of materials, and whether the sorts of things that might be extracted would be useful to sociologists. These comments are easily addressed. Lots of projects produce tons of notes. I recently collected 194 open ended interviews. My antiwar project resulted in dozens and dozens of interviews. We have the volume. Sometimes they are standardized, sometimes not. That’s an empirical issue – how badly does it do with unstructured text? Maybe better than we expect. There is no reason for an a priori dismissal. Finally, I think a little induction is helpful. Yes, we can now pick up sentiment, which is an indicator of emotion, but why not let the data speak to us a little? In other, there’s a whole new world around the corner. This is one step in that direction.
Last week, we discussed Omar’s essay on the end of the “theorist” in sociology. I share the concern of many people who don’t use the label of “theorists.” Theory is often presented in a way that obscure and appears disconnected from the core concerns of sociologists. In the comments, Omar responded to me, and others, by noting that one legacy of Parsons was the creation of the “theorist.” In other words, now that we have theorists, we need to give them something useful to do.
Here is my suggestion: Take a field that is empirically deep and important, but under developed theoretically. Then, work on a synthesis that ties it together and integrates it with current “theory.” Here are my candidates:
- Public opinion
In each case, the topic is hugely important and well developed but even practitioners admit that it is fairly atheoretical. They need your help, theorists. So put that Judith Butler back on the shelf and return that Zizek to the library and show me what you can do.
My former colleague Elizabeth Armstrong and IU alum Laura Hamilton have won the ASA Distinguished publication award for Paying for the Party, their much discussed book on the effects of the party scene on college students. Their book use ethnographic data to describe how the party scene disproportionately affects working class students and mitigates the returns on education. A must read for anyone interested in higher education.
A little while back, Omar released a pamphlet called The End of Theorists. It’s an essay on the state of theory in sociology and some possibilities for the future. Originally given as address to the junior theorist’s symposium, he expanded it into an essay. Omar’s bad news is that the official role of “theorist” has been eroded in sociology. The good news is that one can come up with a new role for theorists that creates a new position for them in the profession. My summary is pithy and leaves out a lot, so I strongly recommend that you read the original.
My comments: First, there’s a conceit in the profession that Omar takes at face value. That we need a separate group of people called “theorists” who do things that other sociologists don’t do. Classically, this wasn’t the case. Max Weber (usually) didn’t do “theory.” He did political economy, though he had some writings that were purely theoretical in character. Durkheim had some purely theoretical texts, like Rules of Sociological Method, but his greatest works were focused on issues like political economy, religion, or social psychology.
So why are these people lumped into “theory?” Very good, or very interesting, answers to important questions have a prolonged impact because future readers try to draw more general lessons.* That fits one common definition of theory – general principles that guide a wide range of cases (e.g., gravity applies to all physical objects, supply and demand curves apply to markets in general). For this reason, I’ve always thought that we shouldn’t have separate theory developers. Instead, we should make our most wide ranging answers into our theory. That’s typically (but not always) how people enter into the “theory canon.”
Second, theory in modern times seems to correlate with some other attributes in sociology – qualitative, history of thought, verbal expression. This can be seen in many ways. For example, people who are heavy in theory tend to do things like historical work, ethnography and culture, which is often but not always qualitative in approach. Just check out the list of speakers for the junior theorist symposiums, or the empirical foci of now classic “theorists” like Bourdieu. Thus, what happens in heavily quantitative areas like criminology, public opinion, or demography has little influence on what the canon of sociological theory should be.
More might be said, but here is what I thought after reading Omar’s essay – The theorist is dead? Good riddance. I’m tired of old books, a balkanized sociology, and posturing. Instead, let’s create theory that distills what is learned from across the profession. That’s a theory that we all can use.
* There is also a political story as well, in that some scholars have big cheering sections while others do not. See Mannheim steamrolled.
Congratulations to Rory McVeigh, Omar Lizardo, and Sarah Mustillo on being named the incoming editors of the American Sociological Review. I wish them all the best and I look forward to being rejected by them!
A few interesting notes: First, if you ever wondered whether blogging damages your career chances, this should put your fears to rest. Second, on Twitter, I asked Omar about the problem of endless R&R’s, rotating reviewers, and other problems that have plagued the current incarnation of the journal. Omar directed me to the proposal that he submitted with Rory and Sarah. A few choice quotes:
- “We aim to maintain that standard, while also seeking out new ways to improve on past performance. We will address the issue of efficiency by strategically using members of the editorial board during the review process and by making a judicious use of the initial R & R decision. Our aim is to use the R & R decision exclusively on papers for which there is strong consensus on the part of the entire editorial team (inclusive of the deputy editors and members of the editorial board assigned to each paper) with regards to potential for publication and the feasibility of the revisions required by the reviewers. We believe that an astute use of the R & R decision will do a lot to improve the efficiency of the review process at ASR.”
- “Our plan is to address this issue by limiting the number of R & R decisions to a maximum of two and by being exceedingly sparing with the practice of granting second R & R decisions. There may be cases where a second R & R decision will be needed, particularly in cases where authors may be somewhat inexperienced and need an additional opportunity to improve the paper or address a critical point. Early editorial intervention, however, should reduce the need for second R & R decisions. However, in no case will we issue a third R & R decision. The decision after a second R & R will be rejection, acceptance, or conditional acceptance. “
Thank flippin’ gawd. The only down side is that I will no longer be banned from the reviewer pool. C’est la vie.