orgtheory.net

Search Results

regnerus’ three mistakes

I was having a conversation a little while ago about scandals in sociology and the topic of of Regnerus’s article came up. That was the article in 2012 that claimed that children of same sex parents had bad life outcomes. My conversation partner suggested that Regnerus’ mistake was in asking the question. I don’t think so.  Asking if there are differences by family type is legitimate and we should be ready to accept less than flattering answers. I also don’t think it is inherently wrong to have biases in research. If he has a favorite answer, then so be it. Long as he doesn’t tinker with the data to get his favorite answer. Nor is problematic to have research sponsors. People with political goals are not barred from research.

What were Regnerus’ mistakes, in my view? He made three:

  1. Making big claims with delicate data. Instead of investing the time to properly locate same sex families and make a good sample, he instead relied on what respondents remembered about family life. Furthermore, scholars who have analyzed the Regnerus data have found that the results may be due to mis-coding cases and data handling errors. In other words, family structures are hard to measure and code properly. And with a small number of cases, small errors can have big effects on the final. This is not the kind of data on which you make big, bold claims.
  2. Politically, Regnerus missed the wave. The big story in American politics is that LBGT equality is arriving.  Polling shows that support for legal same-sex marriage has surpassed 60%. If you’re fighting the new mainstream, you had better be prepared.
  3. Tactically, by rushing the paper through peer review with a friendly editor, it has the appearance of being a hack job. If the paper had been vetted at conferences and undergone a more traditional peer review, then it would have improved.

The issue is that Regnerus was doing something rather ambitious. He was trying to overturn an established research finding, and one with major policy implications. While I don’t agree with his politics, he certainly has a right to do this. At the same time, you don’t take on such a task without seriously thinking things all the way through. That is probably the most basic lesson of all. You should strive toward virtue in your research, especially if you are swimming against the tide.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street  

Written by fabiorojas

June 29, 2017 at 4:30 am

investigating regnerus

I am not sure what this accomplishes, but some journalists are trying to get further records from the University of Central Florida, where the editor of Social Science Research works. From the website of activist and author John Becker:

Despite the wide reach of the New Family Structures Study, much about the process by which it was peer-reviewed and published by the journal Social Science Research remains unknown. We know that the timetable was extraordinarily compressed — according to data from the University of Texas and SSR, Regnerus submitted his paper 20 days before the end of the data collection period and 23 days before the data file was delivered to the university. Sounds fishy, doesn’t it? And the entire process, including the paper’s initial submission, review, revision, and acceptance, took place within six weeks. But why? What are the reasons for moving so quickly? Did Regnerus just catch a lucky break, or is there more to the story? We already know that his funders had an anti-gay agenda and the study itself was plagued by troubling conflicts of interest; were the peer review and publication processes similarly compromised?

Last month, I filed a Freedom of Information Act request with the University of Central Florida, which houses Social Science Research, seeking public records relating to the peer review and publication of the New Family Structures Study. My goal is simply to discover the truth: whether everything was above board and best practices and ethical standards were followed, or whether something more sinister occurred. The documents I requested from UCF may help to answer these important questions.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

April 21, 2013 at 12:09 am

Posted in fabio, research

regnerus, bellesiles, and duesberg

Three scientific controversies worth comparing:

  1. Mark Regnerus publishes a study in Social Science Research claiming that having gay parents is correlated with worse outcomes for children. This is an example of a conservative attacking a belief held by liberals. The subsequent controversy focuses on the actual findings of his survey and the extremely expedited review process.
  2. Michael Bellesiles published Arming America, a book claiming that colonial Americans owned very few guns. This is a liberal attack on a conservative belief.  The subsequent controversy revealed that Bellesiles had almost certainly made up a lot of data, which lead to his dismissal from his university position.
  3. Peter Duesberg is a microbiologist who does not believe that the AIDS is caused by the HIV virus. He believes it is caused by other factors. This, as far as I can tell, not political on his part. He fervently believes in a different hypothesis. There was a controversy which resulted in Duesberg being ostracized by other microbiologists but otherwise retaining is position at UC Berkeley. Duesberg has not changed his opinion, but most other researchers are convinced he is wrong.

Commentary: In academia, you will get attacked if you puncture a widely held belief, regardless of the politics. Somebody will want the credit of taking you down – and that’s not always a bad thing. However, what happens during the controversy is complex. The Regnerus controversy shows that you can survive charges of favoritism and charges of really, really stretching what the data says. The Duesberg controversy shows that you can survive being wrong.* The Bellesiles incident shows that you can’t survive fraud.

A deeper issue is that Regnerus and Duesberg survived because of tenure. They are able to continue teaching and working despite their hugely unpopular opinions because of privilege we give to our senior faculty. However, tenure will only help you though if you play by academic rules. While we may disagree with what these scholars say, I’d chalk up these three example of tenure living up to its promise.

Adverts: From Black Power/Grad Skool Rulz

* There’s a subtle issue with Duesberg. You may not survive if you are associated with a controversial figure. For example, the editor of Medical Hypotheses quit his job after publishing an opinion piece by Duesberg.

Written by fabiorojas

April 5, 2013 at 12:01 am

Posted in ethics, fabio, sociology

regnerus follow up

Think Progress has been digging further into the back story behind the Regnerus/gay parents paper. The news site got one of the study’s funders to admit that the conclusion was predetermined:

Tellez confirmed to The American Independent that he was referring to same-sex marriage cases. In April 2011 — a year before the study was complete — Tellez wrote in a letter that “we are confident that the traditional understanding of marriage will be vindicated by this study as long as it is done honestly and well.” He also suggested that no prior study had properly compared children raised by a mother and father and those “headed by gay and lesbian couples, but of course the Regnerus study doesn’t even do that.

The study was submitted for publication in February 2012 before Regnerus had even completed all of the data collection and accepted just six weeks later, while many other articles published in the same issue took a year between submission and acceptance. Peer review was similarly hurried, with one social demographer admitting that he only had two weeks to review the study and offer a commentary — without even having access to all the data.

Previous Regnerus discussion on orgtheory.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

March 30, 2013 at 12:42 am

comments on regnerus

Finally got around to reading the Regnerus SSR article. A few comments:

1. Up front – my politics: I am against laws that distinguish between people based on sexual orientation. I also believe that people should be tolerant of many sexual orientations, not just heterosexuality.

2. My prior scientific belief: I believe that it may be possible that sexual orientation may be correlated with family outcomes in positive or negative ways, just as there might be differences between other groups (e.g., Latinos may be better or worse parents according to some measure).

3. My prior legal belief: The presence of group differences doesn’t entail policy change unless the differences are extreme. For example, we might discover that alcoholics are worse parents, but I would be against a law that banned alcoholics from having children. In other words, I can believe that some group (e.g., gays) may be better or worse parents than other groups, but that doesn’t mean we should discriminate.

4. What I’ve been told: I am not a family sociologist, but multiple people have told me that prior research tends to find little or no difference between children of straight parents and gay parents. They could be wrong and I’d be willing to update my belief if a sufficiently strong study came out.

5. The actual Regnerus study contains a modest, but interesting result. According to multiple measures, people seem to be worse off if they had a parent who had same gender sexual relations. This isn’t surprising given that most reported two person families were mixed gender. That suggests that same sex contacts were outside the family. In other words, the study measured the “Larry Craigs” of the world. I am not shocked that their children may be worse off in some way.

6. The issue, to me, seems to be the claim that the data provide evidence against same sex marriage. First, even the author admits, there are very few people who reported two parents of the same gender (17, to be exact). Second, there is a severe selection effect. Most of the survey respondents grew up when same sex marriage was illegal, thus preventing what might the equilibrium state in an environment where same gender marriage is legal. To be blunt, the gay people who set up families a decade or more ago are not the same people who might set up families in the current environment.

7. There was petition asking the Social Science Research editor to explain how this paper went through the review process. As an author whose papers have gotten stuck for *years* at a time, I was shocked to learn that it went through in a matter of weeks.

8. Critics claim that the outcry was a “witch hunt” (see the Scatter discussion). That’s a vague and charged term, so I will ignore it. But a few things are safe to say. Science is built on skepticism. If a paper comes out claiming that all previous work in the topic was flawed and produces a controversial result, it would be normal for people to ask questions. The proper response is to provide an explanation of how the research was conducted, not accuse people of a “witch hunt.” No one is asking that anyone be fired or banned from doing sociology. The petitioners are merely asking, “why was this published?”

9. There is some truth to the charge that the outcry is political. Consider a thought experiment, what if a researcher produced a flawed article that supported a liberal policy? Has there been an equivalent level of outcry against bad research that supports liberal points of view? This doesn’t mean that the Regnerus critics should stop. They were right to ask questions. Rather, it means that we should bring the same skepticism to all research, regardless of policy implications. Liberals and conservatives should be equally fearful of sociology’s methodology police.

10. Darren “BMXSherkat was asked the the SSR editor to do an audit of the paper and its review process. Personally, I think this is excessive. The editor, James Wright, is an accomplished scholar and likely knew that the paper would be controversial, even used as ammunition in a political dispute. We give editors great leeway. They may agree with reviewers or override them. He chose to publish this paper after getting some feedback, which is normal. Darren found that the reviewers had some connection with the author. This isn’t always bad. I’m sure that the reviewers of some of my rejected papers know me personally, and that I’ve rejected the papers of people who I like and admire. The bottom line is that the James Wright is an adult and sociology is a contact sport.

11. Bottom line: I think this a modest paper that presents an intuitive result. If one of your parents is gay but is with a different gender partner, then kids may be worse off.  A family where there a sever mismatch in orientation between parents is likely to be stressful, to say the least. At best, the paper would have to be severely rewritten to match the text and the results. At worse, as Darren notes, the paper should just be rejected along with other papers where the claims don’t match the data. The extremely fast publication process suggests that these options were not seriously considered.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

July 29, 2012 at 12:01 am

Posted in fabio, research, sociology

orgtheory’s greatest hits

What are the most commented on posts in the blog’s history? According to WordPress, they are:

  1. The critical realism affair. Technically, Kieran’s critical realism post only got 122 comments, but taken together, the three CR posts got about 160 comments. That was the hardest blogging I ever loved.
  2. Should I stop teaching post-modernism? (144 comments)
  3. Elizabeth Berman’s inequality in the skies. (101 comments)
  4. GRE scores are valid. Sorry, guys. (99 comments).
  5. You know who in Texas. (74 comments)
  6. Brayden and Eszter’s book on online reputation. (74 comments)
  7. How I pick grad students. (63 comments)
  8. Is academia meritocratic? (63 comments)
  9. Steve Vaisey on how to theorize motivation. (58 comments)
  10. World Cup Survey. (57 comments)

Great mix of serious debate on issues ranging from social theory to stratification to social psychology to teaching. Other contenders: Brayden thinks Gladwell is sometimes really, really wrong (54); what has been accomplished with math soc? (51); Kieran discovers that me and one of my PhD students gamed his soc rankings (54); Gabriel Rossman’s infamous “assumptions” post (50); Chris Martin on White privilege (46); a discussion of Jessica Collette’s impostor syndrome research (47); and Chris Winship discusses the ASA amicus brief in the Walmart case (44).

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

September 3, 2015 at 12:01 am

Posted in blogs, fabio, fun

how to fight from a minority position in academia

Over the years, I have been asked by people if academia is hospitable toward minorities. Sometimes, they mean racial minorities or sexual minorities. Other times, they mean ideological minorities in the academy. Once, a student confessed to me that she believed she would be excluded due to her religion (Latter-day Saints, in that case). What all these people have correctly observed is that academia can be an unforgiving place. It’s a place where only half of doctoral students ever finish and only half of those make it into the tenure track. Many spend years working as adjuncts and never get a stable position. The basic truth of academia is that supply outstrips demand, so buyers have leeway to discriminate.

Still, unpopular opinions and people are not always doomed. Rather, it means that you can’t take things for granted. You have to be very careful about how you do things. In fact, it is not terribly hard to find cases where minority people and opinions do well. We can look at those cases and learn. In this post, I want to offer some advice for people in the unpopular position.

  • First, be at peace with the fact that there will be a double standards. While complaint may sooth our feelings, bemoaning double standards is not a productive strategy.
  • Second, fight from a position of strength. Example: James Coleman’s famous report went against the grain in sociology and he was hounded for years. However, he won out because he used the best possible data. In fact, that 1966 data is probably a stronger data set for studying school effects than what many use today.
  • Third, do not fight from a position of weakness. Example: The Gentleman from Texas* decided to fight a contentious battle using very weak data. Result? Two sociologists (IU chair Brian Powell and alum Simon Cheng) found that the data contained serious errors. When the analysis is conducted without data errors, the original conclusions do not follow. Even if Powell and Cheng had not found rather obvious errors, The Gentleman from Texas still had to stretch the data to reach his result. I would not stake my personal reputation on such data.
  • Fourth, cultivate a reputation for mainstream excellence. I have often noticed that people who succeed from unpopular positions are also known as people who have really mastered the mainstream of their discipline. Example: Gary Becker. The original econo-troll said more than enough to ruffle feathers, but no one could question his mastery of traditional economics. You need “street cred.”
  • Fifth, be nice. You will need lots of help to succeed. If you are in an unpopular position, you will need even more than the average. Don’t alienate people with rude behavior. These people will help you later in life. A related point – be useful. If you volunteer in the lab, on the journal board, or in other ways, people will like you and help you back.
  • Sixth, don’t hide, but don’t be a flagpole for the freak show. This is a subtle point. Often, people think there is a dichotomy between the “closet” and “flaming.” That is false. There is lots of space in between. You will be surprised. “Fly causal” and they may lower the shield.

Bottom line: Academia is tough on unpopular people. Be smart, be nice, and you may live to tell the tale.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

* Sorry, I can’t write his name because it automatically attracts this insane commenter who once emailed me to tell me that I was responsible for the murder of LBGT people in the Ukraine.

Written by fabiorojas

May 14, 2015 at 12:01 am

Posted in academia, fabio

top ten orgtheory posts by comments

Written by fabiorojas

June 21, 2013 at 12:11 am

Posted in blogs, fabio

y’know, i’m kind of proud of science right now, even social science

In age of climate denialism and other chicanery, it’s easy to be a science pessimist. But when I stand back, I become a little more confident about things. Science, as an institution, has not buckled under pressure. For example, I think about vaccine skeptics. Truly bad science that has lead to some deaths. However, science did not abandon vaccines and instead went in search of confirmatory evidence and found nil. This was before the retraction of the infamous article in Lancet.

People may sneer at the social sciences, but they hold up as well. Recently, a well known study in economics was found to be in error. People may laugh because it was an Excel error, but there’s a deeper point. There was data, it could be obtained, and it could be replicated. Fixing errors and looking for mistakes is the hallmark of science. In sociology, we often shy away from the mantle of science, but our recent treatment of the Regnerus paper makes me proud. My fellow sociologists obtained the data, analyzed it, and showed that the new data support the long standing finding of no differences between same sex and different sex parents in terms of childhood outcomes.

If you watch the news, the Coburns of the world claim the attention. But when you think about it, the science haters are really standing in the shadow of a much larger enterprise.

Adverts: From Black Power/Grad Skool Rulz

Written by fabiorojas

May 3, 2013 at 12:13 am