Walmart and the ASA (a guest post by Chris Winship)

Note: Chris is a professor of sociology at Harvard University and the Harvard Kennedy School of Government and, since 1995, he has edited Sociological Methods and Research, which is a peer-reviewed scholarly methodology journal. SMR content is also available on the SMRblog.

The current employment discrimination case against Walmart raises the important question of whether social science, and sociology in particular, can effectively participate in court cases and at the same time maintain its scientific integrity. If the answer is yes, there is then the further question of what criteria need to be met for scientific integrity to be maintained. These are important questions requiring discussion, even debate. But first some history.

By early fall, if not sooner, the Supreme Court will make a key decision in the largest employment discrimination suit in history: Dukes v. WalmartOral arguments in the case were heard on March 29. The suit itself, involving a class of as many as 1.5 million women, alleges that Walmart has systematically discriminated against women in its salary and promotion decisions. Potentially, billions of dollars in damages are at stake. The question before the Court, however, is not whether Walmart in fact discriminated against its employees but rather whether such a large case, involving women working in varied circumstances in thousands of different stores and involving different supervisors can be thought to constitute a single class and thus whether the class should be certified.

As reported in a story in the March 27, 2011 issue of The New York Times, the American Sociological Association, as well as many other organizations, has filed an Amicus Brief in this case. (Additional briefs). The primary authors of the ASA brief are Laura Beth Nielsen and Barbara Reskin. The ASA brief is written in support of an expert report submitted for the plaintiffs by sociologist and former ASA President, William Bielby. In a story in the March Footnotes, ASA Executive Director, Sally Hillsman discusses the ASA Amicus Brief at length, pointing out that it is an important example of the type of activity ASA dues pay for. The ASA Amicus brief along with a critique written by three law school faculty members, two of whom are psychologists and all of whom are experienced empirical researchers, will be published in a forthcoming special issue of the journal I edit, Sociological Methods and Research.

A potential question that the Supreme Court may or may not choose to address in Walmart is the admissibility of Bielby’s testimony. Prior to coming to the Supreme Court, the Ninth Circuit Federal Appeals Court (San Francisco), considered by many to be the most liberal court in the country, granted class certification in a 6-5 en banc decision. The minority opinion (page 6255 and following) was quite critical of the scientific status of Bielby’s report. The majority did not comment on this.

In her Footnotes article, Hillsman states that it was critical that the ASA submit an Amicus brief since “whether social science, and sociology in particular, is authoritative and provides valid scientific evidence for helping to define a “class” in class-action cases, and for supporting the contention that social phenomena such as “corporate culture” can and do exist. The implications of this case, therefore, are significant for the discipline.”  She goes on to state: “If the Supreme Court rules in favor of Wal-Mart, the validity of social science research in legal opinions could be significantly diminished.”

The issues that Walmart raises, however, are much broader than Hillsman suggests. It is not whether social science or science more generally should play a role in court cases. Obviously they should. Rather the issue is what are appropriate and effective forms of  participation. In the case of Walmart, the question is whether Bielby’s participation is exemplary as the ASA brief argues or highly problematic.

An expert report is considered admissible if it meets the criteria determined by the Supreme Court in Daubert v. Merrell Dow Pharmaceuticals, Inc. (1993). The basic standard that the court lays out is that expert reports must meet the same standards of scientific methodological rigor as that expected of any published scientific research. (The Federal Judicial Center’s Reference Manual on Scientific Evidence provides a fuller discussion of Daubert.

Although I am not an expert in the Walmart case, I have served as an expert for the defense in more than a half a dozen cases in which Bielby was the plaintiffs’ expert as well as cases in which Barbara Reskin and Donald Tomaskovic-Devey have been experts. In none of these cases have I offered an opinion as to whether a company has discriminated. Rather my sole responsibility has been to evaluate whether in reaching its empirical conclusions the plaintiffs’ expert report meets the standards of scientific methodology. In general, I have found these reports lacking with respect to this standard.

I note that in deposition testimony where Barbara Reskin has been an expert for plaintiffs, she has taken the position that her reports do not meet the standard of science.  For example, quoting from my report in Puffer v. Allstate where we were both experts:

           With regard to whether her opinion is the product of a scientific method, Dr. Reskin stated that her expert report “is not a scientific research product.” (p 383)  Dr. Reskin stated that role “is not that I’m doing or could do what I would see as a scientific analysis of Allstate.  My role is to use my scientific expertise and to draw on social science knowledge to try …to understand what might be going on.   I’m not drawing a scientific conclusion.  I can’t do that in this case.”  (p 77-78)  Also, Dr. Reskin expressed her view that “the scientific method is essentially precluded in a litigation context.” (p 75-76)

                     With regard to whether she would subject her opinion in this case to scrutiny by peer review, Dr. Reskin  stated that she would not hold her causation opinion expressed in Paragraph 2.5 of her expert report up in  front of colleagues and say “this is good science.”  (p 84-85)  Dr. Reskin stated that she “would not dream of  trying to publish Exhibits 2 or 3 or opinions in those reports in an article.”  Exhibits 2 and 3 are Dr. Reskin’s original expert report and her revised expert report.  (Reskin 80, 81)  Dr. Reskin stated that she “would not publish” her causation opinion in Paragraph 2.5 of her report in a refereed journal.  She explained that her causation opinion would not be accepted. (p 84-85)  Also, Dr. Reskin stated that a year from now “I might well have that opinion or I might not.”  (p 85)

(As an aside, I should note that I am well aware that both gender and race discrimination are still a serious problem in this country. In the nearly 20 cases in which I have served as an expert for the defense, my impression, which is qualified since I did no empirical research of my own in these cases, is that in a number of instances the cases had merit, others simply involved extraordinarily difficult individuals on both sides, and others still were without merit at all. I am concerned, however, that law firms, at times, seem more interested in suing companies with deep pockets than in ferreting out egregious cases of discrimination. As social scientists, I would argue that our objective should be to identify places were discrimination is a serious problem and to understand the mechanisms that generate it. More on this in a later post.)

The obvious question that Walmart raises is whether expert reports such as Bielby’s should be admitted in court. I will discuss my position on this particular question on a later post. At a more general and broad level, however, Walmart raises three questions:

  1. How can social scientists appropriately participate in court cases, when it is the courts and parties to a case, not social science that controls the process?
  2. In participating in a court case, how can a social scientist adjudicate between the competing logics of science, built on a notion of consensus, and our courts, which is built on an adversarial process?
  3. Finally and most generally, how then can social science find an appropriate balance between its commitment to objective science and to supporting positive social change?

Appropriate Participation.  For a variety of reasons, it is difficult for a social scientist to act like a scientist in a legal case. There are at least four issues here.

  1. Because of the adversarial logic of the legal process, parties to a case have the incentive to hire experts who will take the most extreme position that is supportive of their side. If an expert is unwilling to do this for a client, the client can simply hire an expert who will.
  2. Court proceedings are often confidential. The vast majority of class action employment discrimination cases never make it to trial, but rather are settled often with a stipulation that all documents be sealed. In the past, Bill Bielby indicated to me that in the 60 plus cases he has been involved in, in only one instance has he testified at trial. Because cases are often sealed, an expert’s scientific colleagues may be totally unaware of the position they are taking. Bielby should be commended for publishing his expert report in Walmart.
  3. Judges are given considerable discretion to decide whether an expert’s opinion meets the standards of Daubert, i.e., essentially whether it is consistent with the standards of scientific methodology. As Bill Bielby has pointed out to me in an email, this can result in wildly different criteria for acceptance. On the one hand, a court may exclude an expert entirely because of problems with one particular data source, say an employee survey. On the other hand, a court may accept a report that provides no description of the methodology used, much less any justification for it. In general, an expert has no way of knowing what specific requirements a judge will have for the admissibility of an expert’s report. Given this indeterminacy, it is very difficult to know what is appropriate.
  4. Because of the legal context, experts typically cannot conduct the types of research that they might otherwise. Companies, for obvious reasons, are not going to allow plaintiffs’ experts to carry out direct research on their firm by say doing employee surveys or participation observation. Even if they did, given that the company was being sued, the validity of any information collected would be questionable.

The above four dimensions create a context which is vastly different in which science is typically done. The parties to a case decide which scientists will be heard, not one’s scientific colleagues as say in journal publication. There is often no opportunity for expert’s views to be know much less criticized in the larger scientific community. The criteria by which a judge will assess any given report is basically unknown. Finally, an expert often cannot collect the type of information needed to formulate a well supported opinion.

Consensual versus Adversarial Logics. Science is by construction a community project in which advancement depends on its members coming to consensus about various theories and facts. As part of this process science requires a practitioner to present evidence that both supports and undermines a position and to carefully weigh the support the evidence provides. In general, “truth” is considered provisional, to potentially be replaced by new “truths” with the acquisition of new data and the development of new and more adequate theories.

Law proceeds on an entirely different logic. It is adversarial, not consensual. Each side is expected to make the strongest possible case for their position without regard to conflicting evidence. It is then up to a jury or judge to then definitely decide, subject to possible appeal, guilt or innocence.

The logics of science and the law are thus in deep tension with each other. As I will discuss in greater depth in a future post, this raises the question of whether experts should play by the rules of science or the adversarial rules found in the law. Daubert is quite clear that expert’s should play by the rules of sciences. However, the de facto practice is that experts generally pursue an adversarial strategy stating their conclusions in the strongest terms and making little if any attempt to qualify them much less to consider inconsistent evidence or alternative explanations.

Objectivity and Progress. Many scientists, and certainly many sociologists, have chosen the careers they have in the belief that scientific knowledge can contribute in critical ways to the making of a better world. This is certainly true in my case. The world, however, seldom changes as a function of science alone. Politics is also often involved.

For science to be persuasive or “authoritative” in Hillsman’s words, the public must believe that its conclusions have been reached by objective means and that they are unlikely to change with time. Otherwise science is simply just one more opinion in the crowded square of public discourse.

Science is faced with three formidable challenges in this context:

  1. The adversarial nature of the court system makes it highly unlikely that scientific experts will be seen as objective. Rather, the public is likely to believe, often rightly so, that experts have been hired because of the positions they take, not because of the quality of their science.
  2. Academics, social scientists in particular, especially sociologists, are much more likely to have liberal political attitudes. (To my mind, Neil Gross is doing the best work documenting this currently.) As such, it can be exceedingly difficult to convince the public that scientific knowledge is driven by objective research as opposed to political motivations. The controversy over climate change is a most obvious example.
  3. Scientific knowledge often changes. In the past, social science has often taken positions it claims as authoritative that have not stood the test of time. As Bill Wilson discusses at length in the first chapter of The Truly Disadvantaged, social science did society no favor when it attacked the Moynihan report (1965) and fully supported the position that a 30% out of wedlock birthrate in the African-American community (nearly 70% by Wilson’s 1987 writing) was unproblematic. Similarly, I have written about how the widely held position in the 1970’s and ‘80’s that mental ability was unimportant to social and economic success resulted in social scientists being ill prepared to address the overblown claims made in The Bell Curve, despite the importance of the issues the book is dealing with.

Society is as uninterested in one-handed sociologists as it is in one-handed economists (As economists are known to say “on the one hand, this seems to be true, but on the other hand perhaps…”). But qualification and tempered assessment are at the core of science. Unfortunately, the courts, as currently structured, undermine this central value in science as well. I would be the first to argue that social science research should play a role in court cases and in public policy making more generally.  In a society where there are value conflicts as divisive as ours, it is critical that there be institutional structures such as science that allow for collective decisions to be informed by fact and not simply swayed by opinion alone. The key institutional design question is how to do this appropriately and effectively.  


Written by gbutler

May 18, 2011 at 12:10 am

44 Responses

Subscribe to comments with RSS.

  1. Dear Chris,

    Fascinating post! Thanks for giving us insight into the frustrating meeting place of sociology and the legal system. I wonder, though, about the model of science assumed by the courts and to which you seem to subscribe. Specifically:

    “Science is by construction a community project in which advancement depends on its members coming to consensus about various theories and facts.”

    I’m not sure this model of science accords well with existing empirical work on the doing of science. While not wholly anarchic, modern science has been shown by science studies research to be far less driven by consensus than perhaps Merton would have liked, and far more adversarial (like the courts). I think by accepting and reinforcing this observably false but normatively desirable picture, scientists (including sociologists) have done ourselves a disservice. When the world expects scientists to have consensus and truth, any revelation of seeming mis-behavior becomes an indictment of an entire research production, and any disagreement becomes reason enough for inaction. The recent history of the tobacco wars and the ongoing struggles over climate change have demonstrated that all too well (see, for example, Proctor and Schiebinger’s Agnotology, Oreskes and Conway’s Merchants of Doubt, Latour’s “Why Has Critique Run Out of Steam?” or Paul Edwards’ A Vast Machine).

    While perhaps not providing much help in the situation at hand, I think that a larger cultural solution to the tension you are describing might entail re-writing our own understanding of the endeavor of doing science, and in turn, re-writing the understanding possessed by the public (and the courts). You hint at this in your closing – “But qualification and tempered assessment are at the core of science.” – but I think might want to go further. Not just qualification and “we’re not 100% sure”, but outright disagreement and dissensus are part of the process. Science is a messy, bloody affair, in some ways like a lawsuit, and I don’t know what good it does us to pretend otherwise. For one alternative take on how to mix science-in-the-making and political decision-making, see Callon et al.’s Acting in an Uncertain World.

    Thanks again for the excellent post!


    Dan Hirschman

    May 18, 2011 at 1:03 am

  2. @Dan, I’d like to distinguish between positive and normative models of science. I’m sure a careful examination of what scientists actually do shows that they deviate from the ideal of open-mindedness and disinterested measured judgment. Don’t you agree that it’s a good ideal though, and that we should attempt to reform the institutions of science to bring reality closer to the ideal?


    Michael Bishop

    May 18, 2011 at 4:08 am

  3. Chris, thanks for the post!

    The jury and adversarial foundations of US jurisprudence have been creaky for a very long time. Trials are games, and parties often have legitimate complaints of unfairness after a game is over, but one can’t replay everything. Persistent biases appear, and inertia, i.e., adversarial forces cancelling each other out, in judicial and legislative forums, keeps biases in place.

    It would seem unlikely to me that science can alter those biases, unless adversarial parties concede scientific validity. In the EU, it may be permissible for judges to assess scientific validity, or for them to delegate such assessments to representatives of professional societies, but I doubt US judges can determine facts, or delegate their determination, unless disputing parties agree to forgo a jury process. Certainly the legislature could impose an arbitration process which is non-adversarial, but in cases such as the Walmart case, the legislature hasn’t done that.

    SImilarly, the legislature could restrict the choice of experts in a given field to members of specific professional societies, but if it doesn’t do so, then adversaries can choose whom they want, and there’s no “loser pays” rule in most cases, so if one side is rich, it can use its wealth to select more prestigious, or better prepared, or just more practiced experts.

    How then ought a professional society or an expert adapt to an evaluative framework which is below the standards of rigor (or respect for peer review) that a professional society or an expert pledge to adhere to? Should the society or the expert play the game, or should they walk away, or should they protest the low standard?

    I think there is an answer. The society or expert should play the game to win, but they should choose the stance they take based on the standards they adhere to in their profession, not those of the court. And they should reward or punish within their own domain based on the stance taken but not based on the standard used in the courtroom. Theater in defense of conclusions based on rigor is no vice.



    May 18, 2011 at 4:20 am

  4. The special issue of SMR looks fascinating and I can’t wait to read all the papers. Thank you for putting that together.

    My feelings on the matter are summed up nicely by the last paragraph of Mitchell, Monahan, and Walker’s paper: “The ASA missed an opportunity to advance sound scientific methods and opinions in litigation when it failed to scrutinize the content of, and method behind, Dr. Bielby’s case-specific claims in the Wal-Mart case. Imagine the impact the ASA could have had by taking a strong stand against flawed methods of the kind Dr. Bielby used. A brief from the ASA rejecting Dr. Bielby’s approach and providing guidance on proper methods and limits for case-specific claims could have reformed the practices of sociologists and other social scientists who serve as expert witnesses. It may be too late for the ASA to play a more positive role in the Wal-Mart case, but we hope that in the future the ASA will promote rigorous practices by sociology experts.”



    May 18, 2011 at 2:14 pm

  5. I have never been involved in litigation as an expert, so I only know when I’ve read. But I think this critique is only helpful insofar as it raises complications that social scientists and others need to consider when they decide whether and how to get involved in such cases. It seems to me the logical conclusion of this line of thought is that social scientists shouldn’t get involved as experts because the process prevents us from doing science they way we’d like to. So, what? Have no social science experts? That seems like a purist – and defeatist – response that doesn’t do anyone any good.

    Why not do a truthful analysis with the information available, then answer the questions as truthfully and completely as possible? If they decide to hire someone else, so be it. Holding social science above the process on principle is wrong.


    Philip Cohen

    May 18, 2011 at 4:54 pm

  6. This has been brewing for some time and I am delighted to see it finally get an airing at Orgtheory. (Anti-dues tea partiers take note, _this_ is what was happening while you were scouring the ASA financial reports.) In my view, Bielby, Winship, and company are perfectly free to fight it out in the courtroom. If you want to be a hired gun, be a hired gun, but do note that there is a downside (as Chris nicely explains). Why the ASA would take a position on this is beyond me. The ASA has no more business filing in support of Bielby in Dukes v. Walmart than it would have had in filing in support of Winship in Puffer v. Allstate. The ASA made a mistake here. Period.


    Als Beruf

    May 18, 2011 at 5:25 pm

  7. Is the existence of this ASA brief fundamentally traceable to the fact that we keep on electing the astronauts of Marxian theory as ASA presidents?



    May 18, 2011 at 7:06 pm

  8. Great post, Chris. I particularly appreciate your framing the issues in terms of the larger question about the role of scientists as expert witnesses. I look forward to the next posts on the amicus brief. I have many questions myself.

    Perhaps the biggest one is trying to square the following circle: (a) Bielby’s report is based on “social framework analysis”; (b) the standard for expert testimony is that the work should use the highest standards of science; (c) the ASA’s amicus brief endorses that standard (unlike Dan, it thinks there are more and less rigorous methods; and unlike Phillip, it’s not just that there is the science that “we’d like to do” and the science that others like to do) and it avers that sociology and other social sciences are rigorous; (d) but then the ASA’s amicus curiae states categorically that “social framework analysis” is not a recognized method in sociology [including in Bielby’s own work] or in any other social science. This leads me to the conclusion that Chris is right that the ASA is essentially admitting that a lower standard of rigor is appropriate for the courtroom. And that implication of course that raises all kinds of questions, not least is whether the ASA would be so comfortable endorsing a lower standard when it comes to cases that do not pursue a liberal agenda, and whether the ASA has fully considered the implications of a public stance that endorses a lower standard (even if it is just in the courtroom rather than in the ASR). Frankly, you have to wonder whether the ASA fully considered the implications of adopting a partisan stance in a case that involves a former president of the ASA, and whether it worried about the apperance of cronyism. To be clear, I’m not saying that there are no answers for these questions. But I haven’t heard them yet.

    BTW, the larger theoretical issues that the Wal*Mart case are fascinating for people who are interested in corporate organization, and the role of decentralization in it. As I understand it, the issue is that Wal*Mart runs its stores in a fairly decentralized fashion when it comes to hiring and promotion decisions, and so therefore it claims that: (a) all women employees cannot be considered a class; and (b) that if there was discrimination, it cannot be considered a corporate *policy*. Bielby’s response is that: (a) Wal*Mart may push decision-making down to the local level, but it has a very uniform *coporate culture* (based on, what I have called centralized control of “voice rights”– i.e., who gets to speak in public spaces [in the name of the corporation]); and (b) the very absence of central controls creates a situation of low accountability, thereby facilitating discrimination. My view is that Bielby’s points are very well taken but that there seems to be no direct evidence of discrimination– and that the very lack of evidence is, at least in part, a result of W*M’s decentralized system! So it’s quite a conundrum. One irony is that if this case wins, it would send a signal that corporations cannot afford to be so decentralized– even though there are many good reasons to operate this way. In a sense, sociologists would be mandating that corporations manage with a much heavier hand!! (And the converse is true. If W*M wins, it might send a signal that corporations that don’t want to have to worry about discrimination should adopt a more decentralized structure).



    May 19, 2011 at 7:46 am

  9. The critical bit from the ASA brief:

    “While we offer no opinion on the substance of Dr. Bielby’s testimony or conclusions, we stress that the methods through which he reached these conclusions are widely accepted and are the bases for research published in the top peer-reviewed social science research journals.”

    The critical fact regarding the ASA brief is that ASA Council unanimously signed off on it. Unanimously. Given the number of drafts that Council requested – described elsewhere by one of the authors of the brief – they can’t plead ignorance. ASA Council unanimously signed off on this brief.



    May 19, 2011 at 11:46 am

  10. Waggaman: Thanks for the quote. That is also hard to square with the ASA’s unwillingess to recognize “social framework analysis” as a legitimate sociological method.

    Do you see the ASA pleading ignorance? (BTW, my understanding is that there were some abstentions or absences from the vote, making it questionable whether the term “unanimous” is appropriate; “with no opposition” would seemingly be more appropriate.)



    May 19, 2011 at 11:51 am

  11. An amicus brief is a friend of the court brief. It helps the court make sense of things. It is entered on a “side” because, as Chris accurately says, that is the nature of the US legal system (adversarial).

    But don’t be fooled. The Mitchell response and the questions raised that discredit all but a few forms of sociology have a “side” as well. You only enter these things on “sides”. The ASA brief says that we are on the side of science and the qualitative methods are part of our science. And it says over and over that we take no position on Bielby’s method (though personally I am willing to take a side on that but it is NOT in this brief). If the case goes forward, there will be a no-holds-barred fight by sociologists if Bielby’s method is sound.

    Unfortunately, our response to Missed Opportunity will not appear in the special issue, but you can read it here:

    The brief was unanimously approved. There were some abstainers at early phases, but the multiple (multiple, multiple) drafts were written and rewrtitten to achieve unanimity precisely so it would be a strong statement in defense of sociology as a discipline.

    Law has sides. Entering the debate at all means you are on one. In this case the defense said the plaintiffs had no case other than the one based on “statistics, sociology, and anecdote.” Well, the defense hired a statistical expert to counter the plaintiff’s stats person. they did not even bother to hire a sociologist hoping to wave it away as silliness. I think we are more than that.

    And, I think it is important to say I do NOT consult in these cases (once, 11 years ago I testified for 3 hours about a particular racial epithet – never about organizations and workplace inequality.)



    May 19, 2011 at 12:44 pm

  12. Laura: Thanks for your post. I will re-check my sources re the abstentions. My understanding is that there were abstentions on the final vote too, but perhaps I am mistaken.

    Otherwise, three questions:

    a. I guess I’m not sure why you think the credibility of sociological research generally, and qualitative research in particular, is on the line in this case. I thought only “social framework analysis” was on the line. Could you clarify why you think this?

    b. You seem to assume that it is obvoius that we should be “entering the debate” and that it is obvious what side we sociologists are on. But: (a) couldn’t we have just sat this out (especially given some clear risks involved)?; and (b) It is well known that a prominent sociologist– Winship– has consistently expressed doubts about Bielby’s approach in these cases. So how is it that the ASA is one side or the other?

    c. Isn’t the decision not to endorse “social framework analysis” awkward for the ASA’s stance that there is no slippage in scientific standards between Bielby’s report and what would be acceptable in a sociological journal?




    May 19, 2011 at 12:56 pm

  13. Vis-a-vis the sarcastic remark about marxist ASA presidents, and the idea that ASA shouldn’t endorse something if a “prominent sociologist” doesn’t agree – or even if the Council decision was unanimous… All that should raise the importance of participating in ASA as a democratic process – including debating over the proper role of our professional association, about which there is no consensus either.

    There is a thread here between Wal-Mart’s “policy” of non-official discrimination (with the implication that unwritten rules and no-paper-trailism prevent class actions) and a hand-off approach to ASA that says it shouldn’t get involved unless the discipline is unanimous and objectivity is preserved. It is not only the legal system that requires side-taking.


    Philip Cohen

    May 19, 2011 at 2:44 pm

  14. Philip: I couldn’t agree more. But whose job is it to encourage/stimulate such debate? I don’t recall any efforts taken by the ASA. Does the ASA membership have to engage in collective action each time it wants to caucus on something?

    BTW, my point wasn’t that the ASA should defer to Chris because he is prominent. My point was just that it is not obvious which side we should be on, and that it would be good for sociologists to understand why the ASA chose one sociological side over another sociological side.



    May 19, 2011 at 2:54 pm

  15. My point (above) was that ASA has no business taking ANY position. The ASA wasn’t called upon by the court to pronounce on the state of knowledge regarding gender discrimination or on current methodologies a la the National Academy of Sciences, it volunteered to write in support of a member’s private consulting work. We hear that ASA had to respond because Walmart lawyers had something less than flattering to say about sociology and didn’t take “us” seriously. Really? (Surely, the authors of the brief are aware that what Walmart had to say is said in courtrooms around the country on an almost daily basis in response to expert testimony?) To make matters worse, there is the practical problem that Walmart will almost certainly prevail at the Supreme Court. Will the end result be that “they” take “us” more or less seriously when all is said and done?


    Als Beruf

    May 19, 2011 at 3:36 pm

  16. I am currently traveling so it is difficult for me to respond at length. I would find it quite helpful if someone could provide an accurate and much more detailed description of how the ASA came to issue its brief and the role that the Council played in it than that in the story in ASA Footnotes. At least two questions seem important.

    First, did the ASA/Council solicit the opinions of anyone who has been critical of Bill’s methodology? I was not contacted. There are many other people who have opposed Bill. Was there any discussion of the considerable criticisms by myself and others that have been made of his methodology?

    Second, was Daubert fully explained to the Council? The SC’s position on admissibility is quite interesting. The question of whether the substance of a report is correct is not to be assessed, only whether the methodology used meets the standards of being scientific. Thus, in an important way when the ASA supports Bill’s methodology they are saying that the report is acceptable.

    All that said, I actually believe that the ASA should take positions in cases, though certainly not all cases by a long stretch. Of course, the question of the process by which it comes to do so is critically important.

    I also would agree with Als Beruf that Walmart may not have been the best place for ASA to spend its political capital. The 9th Circuit, the most liberal in the country by a wide margin, split 6-5. The current Supreme Court is far more conservative than the 9th. Most people are betting Walmart will win. One can make an ethnical argument for ASA’s involvement, but not one based on maintaining our scientific reputation.

    I believe that there is a place for sociological research, both quantitative and qualitative, in the courts. It must, however, meet a higher standard than Bill’s report in Walmart. Those who know me know that I do both quantitative and qualitative work and I have been a strong proponent for decades of methodological (and now theoretical) pluralism and have produced highly successful students who have done both. Rigorous methodology and thinking has been an important part of their success.


    Chris Winship

    May 19, 2011 at 4:41 pm

  17. Well, I guess the question comes down to who defines our discipline? I think of it as a collective effort of our association, the editorial decisions made by peer-reviewers and editors, research processes like verification and replication.

    I do not at all think it is “clear what side sociologists should be on” and anticipate that if the court goes forward, soon Wal-Mart will have a sociological expert on their side. (as it should be in the adversarial system). The sociological experts will go through the process of arguing about whether this is good science (to be clear – this has not happened yet – this is an appeal on a rule 23 motion! we are not even CLOSE to the merits of the case) and the court will decide if it is admissible and probative.

    We have prominent sociologists that are on opposite sides of these things all the time (and, I may add, making some serious dollars doing so). Is that how we want our discipline defined? I don’t. And after lots and lots of conversations, Council voted unanimously (it was an email vote and the bylaws require unanimity in an email vote or there has to be a phone call as I understand it) that the leading people in the field wanted to weigh in with a bigger vision of what sociology is.

    The idea of taking no position in this is kind of nutty — positions are being taken in the name of sociology and I cannot believe so many people think we have no interest weighing in. Do you think of research that you do as science or anecdote? IF you think you do science and if that matters to you, then the brief really should not be a problem.

    @Als – you need to read the brief. We did not support a member’s consulting work. The brief says over and over that we take no position on th equality of the work. Only that it is sociology and sociology is a science.

    If Wal-Mart even HAD a sociologist on their side it would have been much harder because then it would have been Bill vs someone else. But they didn’t. Because sociology is not science. Seriously, that doesn’t bug you?



    May 19, 2011 at 4:43 pm

  18. Did the people who are opposed to the brief actually read the brief? I just read the it, and I’m really confused as to why folks believe that ASA should not have written it. It’s clearly trying to state that sociology is a science that can show certain things, and therefore should not be disregarded in this case. That’s it. It’s not taking a side on the evidence; it’s saying that sociology can provide evidence. It’s saying that sociologists can provide admissible evidence.

    Why shouldn’t ASA speak on whether or not sociology should be considered a really science that is even worthy to be considered in a Daubert analysis, if it gets that far? I certainly want my professional organization to preserve my ability to be legally cognizable. The brief reads to me as simply an assertion that sociology is a science, and then what sociology can show in this case. Like Laura Beth says, it states many times that it is not in support of the report’s findings, but only that, if the report’s author has conducted the research in the ways he has, it could get published in a peer-reviewed journal, as it has been. If the report, which speaks about methodologies that can possibly establish “corporate cultures,” and the possibility of the establishment of such culture is key to surviving a Rule 23 class certification, then it is in sociology’s best interest to file a brief, and have that evidence allowed in a Rule 23 hearing. Otherwise, the court could rule that sociology has no role in establishing such cultures, and status of the discipline takes a large hit.

    The brief is pretty clear that “social frameworks analysis” is a legal framework and not a sociological framework, but it doesn’t discredit it; it says that because it is only a framework, the Court should look to the underlying methods. What is so controversial about that? If the real issue is methods, why get so stuck on the name something is given and not the methods that underlie it?

    In my opinion, I guess obviously as a law & society person, what are we doing this research for if it cannot be useful? This case is not just about Wal-Mart; it is about the future of the use of sociology in law.



    May 19, 2011 at 6:25 pm

  19. I’m sorry, Beth, but a few things that you write don’t make sense to me:

    a. Back to my question [a] above. What is the basis for your claim that sociology is being attacked for not being a science? As far as I can tell, only “social framework analysis” is being attacked, and the ASA brief itself categorically refused to endorse this method as an accepted sociological method. So I don’t get why sociologists should identify so strongly with Bielby’s method (“the science that you do”) such that when Bielby is attacked, we are all being attacked. Is there something that I didn’t read where sociology is being attacked for being junk science? (BTW, the issue of sociology’s status as a science didn’t come up in the– fascinating– SC oral arguments)
    b. I’m not sure what peer review has to do with anything. The amicus brief specifically says that “social framework analysis” does not appear in soc journals. I happen to doubt that a top journal would accept Bielby’s claim that he had demonstrated that Wal*Mart had discriminated against women. If I were a reviewer, I’d say that there is a lot of evidence that is consistent with discrimination, but no hard evidence.
    c. Let’s say the ASA decision was unanimous. Don’t you think that it might have been wise for Council to ask the opinion of Chris and others who have been on the other side of Bielby on many of these cases? And if it did consult people (even non-sociologists wouldn’t have been a bad idea!) who are skeptical about Bielby’s method, what led it to the decision that the ASA should come down on Bielby’s side?
    d. I guess I’m not sure why it matters that there was no sociologist on the other side on this particular case. It is well known that Bielby has been opposed by sociologists on many other such cases. And even if he hadn’t, all because a sociologist is being attacked it doesn’t mean that all sociology is being attacked.

    BTW, I rechecked my source who is in position to know the process by which AsA Council came to this vote (but whose confidentiality needs protecting). S/he reports that at least two people did not vote on the amicus brief. S/he also reports that there was little time to read the report and provide feedback because the debate was conducted on email over a long weekend.



    May 19, 2011 at 6:47 pm

  20. LaToya/gradmommy:

    1. Why do you think that sociology’s status as a science (in the eyes of the law) was at issue?

    2. Why do you think this is true: “if the report’s author has conducted the research in the ways he has, it could get published in a peer-reviewed journal, as it has been.” My view is that the report’s claim that Wal*Mart has engaged in discrimination would be regarded by most sociology journal reviewers as having insufficient empirical support.



    May 19, 2011 at 6:55 pm

  21. Last note for me before I sign off for a few days:

    Why is it so important that these various decisions of Council be unanimous, and that this unanimity be presented to sociologists as if to question it is to be disloyal? The only legislative bodies I know of that are always unanimous on controversial issues are ones where people are afraid of speaking out. I can’t believe that is true for the ASA Council. Would it be so awful if there was some disagreement, and that decisions were taken with some dissenters?



    May 19, 2011 at 7:44 pm

  22. I guess I don’t buy that this is only about “social framework analysis.” Social framework analysis has been used many times in the courts, and it doesn’t seem like it’s even subject to Daubert. It’s used by lawyers, not social scientists. This is an issue not because of the framework, but because of the conclusions. The status of sociology (as a science) is at stake because the use of social science in deciding whether a class can be certified depends on whether the plaintiffs can demonstrate that their individual claims are common enough, and that the named plaintiffs’ situations are typical enough, to be able to proceed as a class. In hiring, firing, and promotion decisions, this is almost impossible to show, in absence of overt discriminatory statements, etc, without sociological data. Therefore sociology is uniquely positioned to provide evidence toward showing how a macro process at the organizational level can have individual effects. If the court rules that this type of evidence is not admissible in these types of cases, not only does it have dramatic effects on class action discrimination cases, but also on how the law sees sociology. If sociology is not seen as a credible science in this extremely important area of the law, it opens sociology up to further critique in other areas of the law. If sociology is not defended and upheld here, lawyers will take that and run with it.

    Regarding the report itself. On one hand you say: “There is a lot of evidence that is consistent with discrimination, but no hard evidence.” But on the other, you say: “My view is that the report’s claim that Wal*Mart has engaged in discrimination would be regarded by most sociology journal reviewers as having insufficient empirical support.” Perhaps as *just* a grad student I’m naive, but much of what is printed in sociology journals are claims that are “consistent” with evidence, but not conclusive; i.e., cannot absolutely rule out alternative explanations. Furthermore, it’s my understanding that the report does not make causal arguments, which is consistent with the majority of peer-reviewed sociology articles. But when the methods are sound, and the theory is sound, and the data is good, we accept the research and conclusions as possible and acceptable, even if we may not agree with the conclusion.



    May 19, 2011 at 10:15 pm

  23. Various people keep claiming here and on Scatterplot that Bill does not make any causal claim. The last sentence in his report is:

    “Discretionary and subjective elements of Wal-Mart’s personnel system and inadequate oversight and ineffective anti-discrimination efforts contribute to disparities between men and women in their compensation and career trajectories at the company.”

    How is the meaning of this different from:

    “Discretionary and subjective elements of Wal-Mart’s personnel system and inadequate oversight and ineffective anti-discrimination efforts are a cause (among possible others) of disparities between men and women in their compensation and career trajectories at the company.”

    Please help me understand why these two statements are not saying the same thing. Clearly, the second statement is a causal claim.

    Correlations (associations) are about things we see in the real world. Causal statements are claims about how we think the world works. They have entirely different epistemological statuses. When Bill says that certain conditions in an organization contribute to disparities, he is making a claim about how things work in the world in general and Walmart in particular.

    If this reasoning is wrong please explain. BTW, Judea Pearl has developed a very neat theory for examining the relationship between observed associations and causal claims based on directed acyclic graphs, basically path models. His approach shows when an observed association supports a causal claim or not. Chapter 3 of Morgan and Winship provides an introduction.


    Chris Winship

    May 20, 2011 at 5:58 am

  24. So I lied about going offline for a few days. But now I really will. Really. I couldn’t resist a response to LaToya/gradmommy’s well-intentioned, seemingly reasonable, but flawed post.

    Here’s the best route into the core issue as I see it. Thanks to Chris, I read the Supreme Court oral arguments (see link in his post). I think they are must reading for anyone who wants to opine about this case and what it means for sociology. See in particular the exchange between the (hapless) lawyer for the “respondents”/plaintiffs, Joseph Sellers, and Justices Kennedy and Scalia on pp. 27-30. A few observations:

    1. The substantive issue that is at hand is a very very hard one, and I submit that few of us could have done much better than poor Sellers in withstanding this line of argument. Is it really a common “policy” when a corporation organizes itself in such a way that it gives broad discretion to line managers but promotes a common corporate culture, such that the employees throughout the corporation can be regarded as a common “class”, and the corporation can be held responsible if/when the managers abuse that discretion (as provided by the common policy) to engage in illegal behavior? And what if the corporation, as Wal*Mart did, has a formal policy against such illegal behavior? Would we say then, due to what we know about organizational decoupling (search on “lip service” in the transcript), that such a formal policy is bullshit and that we should look at more subtle aspects of the corporate culture that is promoted?
    Bielby’s report concludes that the answer to these questions is yes, and he provides no caveats for this view, nor any direct evidence for it.
    (I’ll say more about the weakness of the evidentiary base below. One way to think about the weakness of the evidentiary base: consider the counterfactual where they had evidence that line managers were more likely to engage in discrimination immediately after having gone through a management training program in Fentonville. Do you think they would use such evidence if they had it? And how much priority would they give to that evidence versus the combination of common corporate culture+discretion+implicit bias?)

    The SC justices are skeptical and they are worried about the slippery slope this creates (note that the liberal justices don’t have much to say to help Sellers out; will be interesting to see if they vote. How embarrassing will it be for the ASA if even the liberal justices accept WM’s argument?). They suggest that the implication is that any organization design that pushes decision-making down to line-managers but retains a common corporate culture can be regarded as engaged in a policy of discrimination (so long as implicit association tests show that Americans have unconscious biases). Basically, corporations either adopt centralized bureaucracies or they are guilty. Putting aside the fact that any organizational sociologist worth his salt will tell you that organizations are generally more effective and less alienating when they manage through “indirect” rather than “direct controls” (see e.g., Perrow 1986: 127-129), shouldn’t we sociologists be just as worried about the slippery slope that Kennedy/Scalia et al. are describing? And shouldn’t we be worried that we are signing on to a position that adopts this view with no equivocation and without any direct evidence to support it?

    2. Even if you are not worried about this, do you really think that to be worried about it is so unreasonable as to be equivalent to regarding sociology as junk science? Note that Kennedy, Scalia et al do not say that they are skeptical because Bielby is a sociologist, that he has used sociological methods, or provided (LaToya/grandmommy’s term:) “sociological data.”

    They frame their skepticism in substantive terms.
    (I guess you could argue that this is because Kennedy and Scalia were persuaded by the ASA amicus brief and had to fall back on substantive issues rather than the charge that sociology is junk science. Try *that* counterfactual on for size! Tee hee.)

    And from where I sit, their skepticism is very reasonable I am inclined to agree with Bielby and Sellers that there is indeed no contradiction in saying that WM’s corporate culture provides a great deal of direction to how managers use their wide discretion. But it is a very subtle issue, and I would be very reluctant to say that such direction really is present, and that it “contributes” to discrimination, unless I had more direct evidence– in part because I too am worried about the slippery slope. This may mean that there is no case here. I can live with that because I believe that people and organizations cannot be regarded law-breakers simply because they engage in practices that *generally tend* (and who knows how much– .05 or .95?) to increase the likelihood of law-breaking.

    (Though we may very well decide to eliminate such risky practices through legislation. Alas, our legislative system is horribly broken. Should we then try to push for change via the courts? Perhaps, but then let’s be honest about what we are doing. We don’t actually think we have evidence of discrimination. We are just trying to use the plaintiffs bar to effect changes that we believe will make discrimination less likely. Let’s call a spade a spade.).

    3. LaToya/grandmommy points out that when “much of what is printed in sociology journals are claims that are “consistent” with evidence, but not conclusive; i.e., cannot absolutely rule out alternative explanations.” That is true. But the standard of what is (or really, *should be*; much of what is published does not meet our own standards) acceptable in a sociology journal is very different from what should be acceptable in a courtroom. For instance, much of what is in sociology journals is just theory. And many papers offer new ideas as well as some evidence that helps to bolster the idea that the ideas are useful. We value such papers even though there is incomplete evidentiary basis for fully accepting the theory because: (a) we regard sociology (and science generally) as a collective, cumulative enterprise; (b) we believe that the paper provides potentially valuable new ideas for guiding follow-up research; and (c) we are clear about the weakness of the evidentiary base—i.e., the authors have been clear about the study’s limitations (as reviewers, we demand that authors be forthcoming about such limitations and we reject papers that hide such limitations). I don’t see how any of these apply in cases such as Dukes v WalMart. It is an entirely different enterprise. And Bielby’s report clearly deviates from this model in that states its conclusions with no caveats. It is not a work of sociology, but a work of advocacy. And as such, there is simply no reason why sociologists should feel attacked when this report is attacked.
    A final ironic note: While the amicus brief was apparently motivated by the belief that an attack on Bielby’s report is an attack on sociological methods writ large, I challenge you to read Bielby’s report and say what is distinctively sociological about the methods that he uses. The psychological stuff is obviously not from us. And the stuff on corporate culture is part of the interdisciplinary heritage of organization theory; there is little that is distinctively sociological about it. I guess you could argue that the stuff on accountability has had a lot of sociological contributors (e.g., Petersen and Castilla are names that come to mind) but plenty of others as well (e.g., Tetlock). So I can see that a sociologist is being attacked (for a work of advocacy, for which he is highly compensated). But in what way is sociology being attacked? I don’t see it. And I tend to share the skeptics’ worries. Am I then a traitor to the sociological cause?



    May 20, 2011 at 8:00 am

  25. His approach shows when an observed association supports a causal claim or not.

    Assuming a particular ontology of the causal relation, ahem.

    (Chris is of course right that Bielby is clearly arguing that “Discretionary and subjective elements of Wal-Mart’s personnel system” causally contribute to disparate compensation between men at women in the organization. If he weren’t, there would be no point having him as an expert witness for the Plaintiffs.)



    May 20, 2011 at 1:21 pm

  26. The discussion of causal analysis in Bill Bielby’s report on the Wal-Mart case has lacked attention to an important aspect of the report: the review of employment statistics on Wal-Mart compared to both itself and its competitors. Wal-Mart has a promote from within human resource practice and male non-managers are much more likely than women to move into managerial ranks. But might this reflect different labor supply choices of women and men? Well it turns out women are significantly less likely to become managers at Wal-Mart than women in other big-box retailers in the same localities. There is a basic regularity of gender disparity, which rules out two important aspects of labor supply. Differences in labor supply are the most generally recognized sources of non-discriminatory gender employment disparity. Bielby’s report examines both theory (implicit bias) and evidence (the absence of managerial accountability for bias in hiring and promotion) to explain why this disparity exists. This is consistent with typical social science causal reasoning that is published in our journals. If the Bielby report went through a peer review process, reviewers may have asked for him to be more explicit about alternative explanations for the results (but they also may not have).

    This is not to say that a report for class action is the same as science, it is clearly not. The adversarial nature of legal proceedings typically leaves speculation about the error term in an analysis up to the other party in the case. I have been an expert in a few of these cases and have found this aspect of the process difficult to reconcile. My solution has been to raise alternative explanations where they seem plausible because there is evidence to support them, but not to speculate wildly as reviewers often ask us to do as part of the journal submission process.


    Don Tomaskovic-Devey

    May 20, 2011 at 2:55 pm

  27. Don: Thanks for your post. It is helpful. I don’t necessarily disagree that it’s possible to publish a paper in ASR or AJS with the kind of approach that you describe– there are definitely papers that are supported by much more tenuous theoretical and empirical support– though I’d disagree that such an analysis would warrant publication. Putting that aside though, I’m curious though about the question that I have repeatedly posed to Laura: Do you agree that the scientific status/legal credibility of sociology was at stake in this case? My post above spells out my reasons for why I don’t see it this way.



    May 20, 2011 at 3:09 pm

  28. Ezra
    You ask: “Do you agree that the scientific status/legal credibility of sociology was at stake in this case?”

    My answer: “Yes”

    If the supreme court effectively ruled that social scientists reasoning from general theory and incomplete but case specific evidence to explain some social outcome was not scientific, than we could be excluded by precedent from speaking to the facts of a case in almost any context.

    Now, Chris’s earlier points about the design of evidence in adversarial legal contexts being less than ideal has some merit. If the supreme court was to rule that the current practice of discovery did not yield ideal data, that discrimination was so important that we should do better, and went on to mandate that impartial social scientists should be hired by the court to design and implement as close to an ideal study as possible and employers be compelled to provide all requested data, that would be better than current practice. But the legal trend since the mid-1980s has been to insulate companies from such law suits, so such a ruling seems unlikely in the extreme.



    Don Tomaskovic-Devey

    May 20, 2011 at 4:35 pm

  29. yes I am quite convinced that one of the questions in this case was whether sociology is a science. I am convinced of that, in part, because lawyers on both sides were arguing about it in their briefs. And the briefs of all parties are page limited so that space is valuable.

    Also, watch what happens — even if THIS class is not certified (and if I had to bet, I would bet these women are going to lose on Rule 23), the case will go on with smaller classes (women in this particular store or region or women who applied for management positions or something like that). And then, there will be sociologists on both sides. Soon there will be a guest post here by Amy Myrick that goes more into this.

    There are SO MANY questions, I am trying to reply to them when I can and do my work too. I’ve spent TONS of time on this already and you may note, I am not even an author on the brief (and NONE of my own work is cited), my reply has no published home, so my service to the organization (whether you agree with it or not) has been a significant time commitment and I am trying to write a book. (interestingly enough, on employment civil rights).

    On that note, for people interested in the substance of the Wal-Mart case and how class action works or does not work, we had an interesting paper that did a nice job looking at outcomes in these kinds of cases rejected (in the journal of empirical legal studies) It shows that plaintiffs on their own almost never win employment discrimination cases. Would I say being a solo plaintiffs CAUSES them to lose? No. But they are far more likely to lose. far, far, far more likely to lose almost no matter what (no matter what the EEOC said about their case, no matter what the claim is — race, sex, age, pregnancy, etc). Class action status (along with other forms of collective mobilization like a disparate impact claims, and public interest law firms) are how civil rights enforcement in this country works. the stakes here are high for sociology and for workers (read the amicus filed by the Institute for Women’s Policy research available here: if you are interested in research on this part of things.

    Just to be fair minded, I should post some D side amicus briefs — here is the washington legal foundation’s:

    Oh look! If you read section 3, they are arguing the same thing we are — The sociology should have been tested for scientific reliability under Daubert at the trial level. And here is the defense bar’s brief:

    here is a list of everyone that filed:

    Sep 24 2010 Brief amicus curiae of Intel Corporation filed.
    Sep 24 2010 Brief amicus curiae of Chamber of Commerce of the United States filed.
    Sep 24 2010 Brief amicus curiae of DRI – The Voice of the Defense Bar filed.
    Sep 24 2010 Brief amicus curiae of Equal Employment Advisory Council filed.
    Sep 24 2010 Brief amicus curiae of Washington Legal Foundation filed.
    Sep 24 2010 Brief amici curiae of Altria Group, Inc., et al. filed.
    Sep 24 2010 Brief amicus curiae of Pacific Legal Foundation filed.
    Sep 24 2010 Brief amicus curiae of Retail Litigation Center, Inc. filed.
    Sep 24 2010 Brief amicus curiae of California Employment Law Council filed.
    Oct 21 2010 Brief of respondents Betty Dukes, et al. in opposition filed.
    Oct 29 2010 Reply of petitioner Wal-Mart Stores, Incorporated filed.
    Nov 1 2010 DISTRIBUTED for Conference of November 23, 2010.
    Nov 29 2010 DISTRIBUTED for Conference of December 3, 2010.
    Dec 6 2010 Petition GRANTED limited to Question I presented by the petition. In addition to Question I, the parties are directed to brief and argue the following question: “Whether the class certification ordered under Rule 23(b)(2) was consistent with Rule 23(a).”
    Dec 23 2010 SET FOR ARGUMENT Tuesday, March 29, 2011.
    Dec 29 2010 Consent to the filing of amicus curiae briefs, in support of either party or of neither party, received from counsel for the petitioner.
    Jan 12 2011 Consent to the filing of amicus curiae briefs, in support of either party or of neither party, received from counsel for the respondents.
    Jan 20 2011 Joint appendix filed. (Jt.appendix in 3 volumes) (Statement of costs filed).
    Jan 20 2011 Brief of petitioner Wal-Mart Stores, Incorporated filed.
    Jan 25 2011 CIRCULATED.
    Jan 25 2011 Brief amicus curiae of California Employment Law Council filed. (Distributed)
    Jan 26 2011 Brief amicus curiae of Pacific Legal Foundation filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Intel Corporation filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Association of Global Automakers, Inc. filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Chamber of Commerce of the United States filed. (Distributed)
    Jan 27 2011 Brief amici curiae of Society for Human Resource Management and HR Policy Association filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of International Association of Defense Counsel filed. (Distributed)
    Jan 27 2011 Brief amici curiae of Altria Group, Inc., et al. filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Costco Wholesale Corporation filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Equal Employment Advisory Council filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Securities Industry and Financial Markets Association filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Retail Litigation Center, Inc. filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of Washington Legal Foundation filed. (Distributed)
    Jan 27 2011 Brief amicus curiae of DRI – The Voice of the Defense Bar filed. (Distributed)
    Jan 27 2011 Brief amici curiae of Atlantic Legal Foundation and New England Legal Foundation filed. (Distributed)
    Feb 4 2011 Record received from U.S.C.A. for 9th Circuit. (1 box)
    Feb 20 2011 Record received from U.S.D.C. for Northern District of San Francisco. (12 boxes)
    Feb 22 2011 Brief of respondents Betty Dukes, et al. filed. (Distributed)
    Feb 25 2011 Brief amici curiae of Labor Economists and Statisticians filed. (Distributed)
    Feb 28 2011 Brief amicus curiae of American Association for Justice filed. (Distributed)
    Mar 1 2011 Brief amici curiae of U.S. Women’s Chamber of Commerce, et al. filed. (Distributed)
    Mar 1 2011 Brief amicus curiae of Law and Economics Professors filed. (Distributed)
    Mar 1 2011 Brief amici curiae of Public Justice, P.C., et al. filed. (Distributed)
    Mar 1 2011 Brief amici curiae of Consumers Union of United States, Inc., et al. filed. (Distributed)
    Mar 1 2011 Brief amici curiae of National Employment Lawyers Association, et al. filed. (Distributed)
    Mar 1 2011 Brief amici curiae of American Sociological Association, et al. filed. (Distributed)
    Mar 1 2011 Brief amicus curiae of Institute for Women’s Policy Research filed. (Distributed)
    Mar 1 2011 Brief amicus curiae of Public Citizen, Inc. filed. (Distributed)
    Mar 1 2011 Brief amici curiae of United Food and Commercial Workers International Union, et al. filed. (Distributed)
    Mar 1 2011 Brief amici curiae of NAACP Legal Defense and Educational Fund, Inc., et al. filed. (Distributed)
    Mar 1 2011 Brief amicus curiae of Civil Procedure Professors filed. (Distributed)
    Mar 1 2011 Brief amici curiae of American Civil Liberties Union and National Women’s Law Center, et al. filed. (Distributed)
    Mar 17 2011 Reply of petitioner Wal-Mart Stores, Incorporated filed.
    Mar 29 2011 Argued. For petitioner: Theodore J. Boutrous, Jr., Los Angeles, Cal. For respondents: Joseph M. Sellers, Washington, D. C.



    May 20, 2011 at 4:40 pm

  30. Surely, Don, that is an argument about sociology’s role in the courts, not sociology’s primary mission of teaching and research. Right?



    May 20, 2011 at 4:50 pm

  31. Yes, I am talking about the courts and sociology. There are interesting parallels in terms of methods.


    Don Tomaskovic-Devey

    May 20, 2011 at 5:18 pm

  32. […] case and is a coauthor with Laura Beth Nielsen on related papers.  The following is a response to Chris Winship’s earlier post about the Amicus […]


  33. […] Chris posted “Walmart and the ASA” to the orgtheory blog. The piece is about the recent ASA Amicus Brief which supports the use of social framework analysis by social scientists who act as expert witnesses in, and whose conclusions can support class certification of, large class action discrimination litigation cases. This issue is generating a lot of debate, on scatterplot as well as orgtheory, the outcome of which might have important implications for the role of social scientists in legal processes, such as the current Walmart case [more on the case here 1, 2, 3] to be decided on this summer. Discussion on the blog has also incorporated other issues relating to the mission and role of ASA. Chris’ post refers to articles written in response to the Brief, and additional articles are currently going through peer review at Sociological Methods & Research (the journal Chris edits) for inclusion in a special issue of SMR that is expected to see print in the winter of 2011-12. Check out the SMRblog also for updates and information on the Special Issue, tentitively titled “Inferring from the General to the Specific: What Happened? Why?”. Check back over the summer for updates–this special issue material will definitely appear online before seeing print, to start, Greg Mitchell’s paper [link is to draft], in response to the ASA brief will be available through Sage sometime in June, if not before. […]


  34. […] box and is a coauthor with Laura Beth Nielsen on related papers.  The following is a response to Chris Winship’s progressing post about a Amicus […]


  35. Laura: Thanks for answering my question. Unfortunately, I do not find your answer to be persuasive. I simply see nowhere where sociology was under attack. Just the expert testimony of a particular sociologist. And in a reasonable, substantive manner.

    It appears from your response and those of others that the basis for your contention that sociology was under attack is that the petitioner/plaintiff’s brief said that Bielby’s expert testimony was based on “statistics, sociology, and anecdotes.” There seem to be two possible ways of construing this as an attack on sociology, but both are unviable.

    The first seems to be something like what is given in ASA Executive Officer Sally Hillsman’s note in the March 2011 footnotes (
    “The ASA amicus curiae brief tries to persuade the Supreme Court that the use of sociological research is valid evidence in this case and not, as Wal-Mart has argued, “anecdote” and mere “statistics” that should be excluded from considering whether there is a “class” of plaintiffs as well as the merits of case.”

    But this is an obviously erroneous interpretation of what the petitioner’s brief says. It doesn’t say that sociology is “’anecdote’ and mere ‘statistics.’” Rather, it says that Bielby’s testimony is based on “statistics, sociology, and anecdotes.” That is, statistics is a separate category from sociology, which in turn is a separate category from anecdotes. A key implication of this: if you think that the scientific status/legal credibility of sociology is under attack, you must also think that the scientific status/legal credibility of STATISTICS is under attack!

    So this is obviously an unviable interpretation and it is extremely unfortunate to say the least that the only communication that ASA members have received from the ASA about the rationale for the amicus brief is based on this clearly erroneous interpretation of the petitioner’s brief.

    The second possible basis for saying that sociology was under attack is by looking at what the petitioner’s brief actually says about “sociology.” Note first how the phrase “statistics ,sociology, and anecdotes” actually functions in the petitioner’s brief—these are *headings* for three “categories of evidence.” The brief says that Bielby relies on each of these types of support and those three words that they have chosen as titles for the kinds of evidentiary support that Bielby is relying on. Ok, so maybe that means that the petitioner is saying that these “categories of evidence” have no legal credibility?

    Well, let’s look at all three of these categories. Do the petitioners say that “statistics” has no legal credibility? It is notable that the American Statistical Association clearly does not think so (they do not show up in Laura’s list of amicus briefs; a group of “Labor Economists and Statisticians in Support of Respondents” did write a brief–, and it is a well-reasoned rebuttal to the petitioner’s argument in the “statistics” section. Nowhere in that brief does this group feel the need to defend statistics or econometrics as a field). And with good reason. If you look at the petitioner’s brief, you will see that its problem with “statistics” is not a categorical objection to statistics or statistical analysis, but merely an objection to the statistical analysis provided by the plaintiffs’ expert statistician, Dr. Richard Drogin (here is his report, btw: Let me underline the point that they do not attack the field of statistics or the credibility of Drogin himself. They merely present a substantive objection to the particular kind of statistical analysis that Drogin performed, arguing that it is insufficient to support the plaintiffs’ argument that the Wal*Mart women constitute a common class.

    And if you read the “sociology” section, you will find the same thing. Nowhere do they attack sociology as a discipline, nor do they even attack Bielby’s credibility. In fact, here is an incredible fact: the word “sociology” does not even appear in the body of the section in which the credibility of sociology is supposedly under attack! Pretty strange way to attack the credibility of something by not even mentioning it by name, no? But this puzzle dissolves once you realize that they are in fact not attacking sociology. Rather, they merely voice substantive objections to Bielby’s arguments along the same lines that reappear in the Supreme Court oral arguments and which I summarize in my earlier comment ( So just as they do not attack statistics as a field or Drogin, they also do not attack sociology or Bielby. Rather, they argue that the sociological analysis performed by Bielby (in fact, I pointed out in that comment that there is nothing distinctively sociological about Bielby’s analysis, but since Bielby presents himself as a “sociology expert” the petitioners reasonably infer that the kind of analysis in Bielby’s brief is “sociology”) does not support the plaintiffs’ arguments that the Wal-Mart women form a class, and their argument is quite substantive and reasonable. Moreover, and as I pointed out in that earlier comment, the Supreme Court oral arguments clearly show that sociology’s credibility was not at issue.

    And finally, I simply do not know what Laura is seeing in the Washington Legal Foundation’s brief on behalf of the defense/petitioners ( that supports her claim that such friends of the defense are attacking sociology. The word “sociology” appears once—to characterize Bielby as a “sociology expert.” And what is it they don’t like about Bielby’s testimony? Check it out (pp. 10-11):

    “expert opinion is not admissible simply because it is based on reliable principles and methodology. The application of that methodology to the facts of the case must be reliable. See Fed. R. Evid. 702(2)-(3). For this reason, whether an expert’s opinions were deemed admissible in one case has little bearing on whether such testimony should be admitted in a subsequent case where the expert seeks to apply his methodology in a new, untested way. Although the expert in Price Waterhouse v. Hopkins used a methodology similar to Dr. Bielby’s, she did not seek to apply her opinions to thousands of employees spread across the country in a nationwide class.”

    So these guys are in fact conceding that Bielby’s analysis is based on “reliable principles and methodology”!!! Their beef is merely with how he has applied these “reliable principles and methodology” in the present case and to the specific issue of whether the W*M should be regarded as a class.

    So where-o-where is the attack on sociology? And how did this amicus brief get out without: (a) consultation with Chris or other sociologists with relevant expertise who are known to have doubts about Bielby’s methodology; (b) consideration for what it means to endorse expert testimony (which has all the deviation from scientific research noted both by Chris and by Don) as a piece of sociological research; (c) sufficient vetting such that no one caught damaging statements such as that sociologists think that causality can sometimes be inferred from correlation? [see here:;%5D and(e) due consideration of the risks of being seen to support a former ASA president in a case (and his larger expert-witnessing franchise) for which he was (is) highly compensated? And then there is the question of why the amicus brief was presented to ASA members in erroneous, tendentious terms.

    If the ASA has indeed now committed itself to transparency, it should organize a panel at the ASA meetings in August at which these and other questions can be discussed, with participation from participants in the process behind the amicus brief as well as relevant sociologists such as Chris who should have participated. What possible reason could there be for not holding such a panel?



    May 21, 2011 at 8:12 pm

  36. Two corrections for the above:

    a. “petitioner/plaintiff” should read “petitioner/defense”

    b. The petitioner’s brief did not say that “Bielby’s expert testimony was based on “statistics, sociology, and anecdotes” but that the *plaintiffs’ case* was based on “statistics, sociology, and anecdotes.” Drogin’s report contains the statistical analysis they challenge, Bielby’s report contains the sociological analysis they challenge, and then they challenge some anecdotal evidence. If anything, this correction strengthens my point that sociology itself was in no way under attack– just the appropriateness of the analysis (which is arguably distinctively sociological) performed by Bielby. And as I have emphasized, it was critiqued in a very reasonable, substantive manner. (By contrast, the attack of the statistics section seems less cogent, and is effectively parried by the amicus brief of the group of labor economists and statisticians). The scientific status/legal credibility of sociology was in no way on the line here.



    May 22, 2011 at 8:40 am

  37. […] the debate about the ASA amicus brief is a fascinating theoretical question about the link between organizational culture and […]


  38. […] last night reading through all the comments on orgtheory and scatterplot. My key goal in writing my initial post was to get a discussion going about the role of sociology in the courts and the particular problems […]


  39. […] has been discussed about the Walmart case and ASA Amicus Brief in the postings and comments on the orgtheory [with subsequent posts 1, 2] and scatterplot blogs. Little, however, has been said about the […]


  40. […] amicus brief (pdf) in the Wal-Mart case – subject of so much lively debate over on orgtheory and scatterplot about whether the ASA was right to file the amicus brief, whether it was a good […]


  41. […] rejected by the Supreme Court today. I’ve nothing much to say about the case or the role of sociologists that has become something of a flashpoint this […]


  42. […] Supreme Court has sided with Wal-Mart in the class action case. As regular readers of this blog will be well aware, sociologists have been more than usually involved in the case and the debate […]


  43. […] brief intervening in the Wal-Mart class-action discrimination case. See it at Orgtheory here, here, and here; and at Scatterplot […]


  44. Wow, this paragraph is pleasant, my sister
    is analyzing these kinds of things, so I am
    going to inform her.


    lungs cancer

    August 3, 2012 at 5:30 am

Comments are closed.

%d bloggers like this: