charismatic organizations: the case of alcoholics anonymous
The media covered a new book by Lance Dodes called The Sober Truth. In the book, Dodes surveys the evidence on rehab and finds that there is literally no evidence that rehab, AA or other popular methods for kicking drugs are effective. From a recent Alternet article:
Peer-reviewed studies peg the success rate of AA somewhere between 5 and 10 percent. That is, about one of every fifteen people who enter these programs is able to become and stay sober. In 2006, one of the most prestigious scientific research organizations in the world, the Cochrane Collaboration, conducted a review of the many studies conducted between 1966 and 2005 and reached a stunning conclusion: “No experimental studies unequivocally demonstrated the effectiveness of AA” in treating alcoholism. This group reached the same conclusion about professional AA-oriented treatment (12-step facilitation therapy, or TSF), which is the core of virtually every alcoholism-rehabilitation program in the country.
What I find interesting is that I was told this before by physicians and social workers. These programs work for very few people and this is common knowledge. But why didn’t I draw the logical conclusion? If it’s expensive ($200,000 for a stint in a fancy rehab center) and it doesn’t work, why not just stop doing it?
Two answers: The Robin Hanson answer is that it’s a signal of morality. We do it to show that we care, even if the evidence is dodgy. Another (not unrelated) answer is that charismatic orgs get less scrutiny. AA is trying to be nice to people and help them overcome serious problems, so I am less inclined to search for evidence that assesses their effectiveness. This is different than, say, a think tank that is pushing a policy that I don’t like. Then, I’ll search high and low for all the evidence I can find to fight them.
Bottom line: We should probably get tougher on organizations that claim to do good. We’re probably giving out too many free passes.
50+ chapters of grad skool advice goodness: From Black Power/Grad Skool Rulz
orgtheory.net 
Before we throw AA under the bus – to what other programs are these studies comparing AA? What is the control group in an AA study? Or is the 5-10 percent recovery rate equivalent to having an alcoholic not join any group at all?
LikeLike
brayden king
April 21, 2014 at 1:33 pm
AA is a cult. An anonymous higher power, a much more powerful demon drink, and the twelve steps/commandments.
They aren’t comparing AA to anything- it just sucks that bad in terms of actually getting things done.
I’ve known this for years, but people will just repeat the phrase, “it works if you work it” back at you if you point to the statistics. Total brainwashing cultish crap. Sadly, this kind of awfulness is working its way into everything.
LikeLike
August
April 21, 2014 at 1:49 pm
@brayden: A 5-10% success rate for ANY policy is considered paltry. Perhaps the best way to think of it is hedonically – how much would you pay for a medicine that cures you 5% of the time? The answer is probably: “Zero, unless I was desperate.” The article also notes that there seems to be a non-trivial percent of people who stop using drugs without any intervention. Thus, it seems as if this really is a bum deal. AA is relatively low costs, but rehab has huge costs.
LikeLike
fabiorojas
April 21, 2014 at 3:41 pm
Addicts and the addiction treatment community do not avoid criticizing 12-step programs like AA, as August has already demonstrated.
LikeLike
Jenn Lena
April 21, 2014 at 4:58 pm
Agree with Brayden here, jumping to conclusions a bit. Some things to consider: how do we define success of an alcohol treatment program? Is it merely in terms of relapse? Over what time horizons? What about potential social benefits of treatment beyond relapse? What about success relative cost of the program?
Second, as Brayden notes, what are the alternatives and how well do they work? More research is needed. While there are emerging best practices, I’d be hesitant to throw out any program without more research.
Third, as there are with most programs/policies, there are many different ways that AA and other treatment programs are implemented. So to think of AA as one thing with a very rigid structure that is implemented consistently across venues is just wrong. Therefore, questions inevitably arise about the extent to which any program is implemented with fidelity to its intended design and if/when it is, what are the results? This is an empirical question. Though not one that is easy to research.
Not to mention that programs might work for some types of people and not others. So their effectiveness is contingent upon where and with whom the program is applied to. AA might work for some people and not others. It’d be important to try to tease out where it might work and where it might not. Again, this requires more research,
I don’t think we have enough evidence to throw any of the treatment options out. But I do agree that we need to evaluate programs and their effects. However, it should be noted that there are a lot of people in the field of addiction treatment who are doing this already. So it’s probably wise to consider the research.
And if we are to get tougher on organizations, we should be aware that in order to do evaluate the effectiveness of treatment options, more money will need to be spent to fund the research necessary. Easier said than done. And in the case of AA, there are a whole host of issues with doing the research: problems of self-selection; the creation of control groups; access to sites for research, anonymity of subject, etc.
LikeLike
SD
April 21, 2014 at 6:01 pm
For a good summary of the emerging best practices, see:
Click to access RBBPP.pdf
LikeLike
SD
April 21, 2014 at 6:06 pm
“Zero, unless I was desperate”. I am not sure how to interpret that sentence. I would have thought it obvious that the families of these alcoholics are quite desperate? Have several friends who lost a parent (to alcohol and suicide) in this way. I assure you they were quite desperate and a 10% chance sounds quite fantastic. 10% is higher than I would have guessed, so I am quite impressed.
LikeLike
Anonymous
April 21, 2014 at 7:51 pm
Anonymous: In general, when I think about policy and trade-offs, I think of medical situations in comparison. Expensive treatments with cure rates of 5%-10% are usually discouraged by most physicians. You would only use them if (a) other treatments have been used and (b) the patient is terminal or in extreme pain.
I am not a health care professional, so I can’t assess (a). (b) is tricky because addiction is horrible but there is a wide range of experience and behavior. If these programs were relatively short in duration and low in cost, then it would be reasonable to make AA and related programs the default. No harm in spending modest resources on uncertain outcomes. But that is not the case. You can easily spend a sum equal to a new home or a college education or a few years of retirement. It’s a trade off that needs to seriously assessed.
LikeLike
fabiorojas
April 21, 2014 at 8:28 pm
Of course all this has to be seriously assessed, agree. Also agree on the overall call for more critical research on do good organizations with such wide-ranging (lack of?) impact. But daughters and sons of alcoholics are often very desperate and I would certainly also cut 2 years of my retirement for a 10% chance of rehab (if that includes the serious cases that might lead to suicide). Easy choice – if it’s the best option out there.
LikeLike
Anonymous
April 21, 2014 at 8:48 pm
Anonymous: I can’t dispute a person who openly accepts the risk. As an empirical matter, though, it is worth asking if people confronted with the choice do actually make that trade off. I think it would depend a lot on what the relationship is.
How many people, for example, take out mortgages to pay for their own rehab with a 10% success rate? The situation is different with parents. An older parent paying for a child’s rehab may be engaging in a wealth transfer that would soon happen anyway. But, for example, how many people would forgo a college education to help a sibling by giving them a 10% chance of long sobriety?
The empirical pattern of these choices would give us insight into how much we’d actually value these types of therapies with their estimated success rates.
LikeLike
fabiorojas
April 21, 2014 at 9:01 pm
The success rates for all kinds of addiction treatment are bad. In fact if this data is correct, more people “naturally” recover without intervention. I have conducted ethnographic research on addiction treatment, comparing a program in the healthcare system to one in the criminal justice system. Although this is not the kind of research that examines success rates, I can provide some ideas about why we keep using it despite the uniformly bad results. The most important answer as to why we keep doing this is that many institutions push/force people into rehab. The expansion of the criminal justice system in the last 35 years has not just expanded prisons, it expanded treatment. The CJ system is the largest single source of treatment admissions. It refers people to AA/NA as well as formal rehabs. Workplaces, TANF, and child welfare agencies also refer people to treatment. State funding, especially CJ funding, will only support certain types of programs, and a large majority of programs rely on some kind of government money or Medicaid. So rehab continues as is because we are asking it to solve a lot of problems that are only partially related to substance use.
In addition, most rehabs follow the 12-step model derived from AA, so there isn’t much variation in the first place. Even ones that do not officially use this model adopt many of the basic ideological tenants and practical techniques. It can be quite challenging to find treatment that is not 12-step based and nearly impossible to find a rehab that is not abstinence-only. (Still, the alternative models do not have very good success rates either.) For long term support, most people have no where to turn but AA/NA because health insurance will not fund more than about 4 weeks of rehab and limits outpatient visits. Moreover, AA approaches dominate because most people who work in the field are “in recovery” themselves and chances are they went through a 12-step program. AA seems particularly good at inspiring loyalty and single-minded devotion. Plus, substance abuse counseling is a para-professional field, which means prospective counselors are not subject to the same kinds of professionalizing forces as medicine or psychology or social work. Becoming a counselor usually requires a high school diploma and state certification based on some education credits, a test, and clinical experience. I found the field to be resistant to medical/scientific incursions into their turf and to value counselors who work mostly from personal experience. This is not necessarily a bad thing, but it does make addiction treatment unusually closed to research and to trends in other mental health fields.
I’d like to second SD’s question on what counts as “success.” These studies almost always measure success as not being rearrested or relapsing. But I found the programs are trying to do so much more than getting people to stop using. The people who work in these programs would never think of success in the way these researchers do. They tend to see relapse as normal and inevitable. Programs have very ambitious goals of making people morally good or self-actualized, but they are also hoping to change how people cope, get clients to resign themselves to their lot, improve their social relationships, reduce unhappiness, provide insight into one’s self. The programs may actually do these things, even if people relapse.
Finally, programs and AA/NA always blame relapse on individual failure, rendering their treatment methods nearly impossible to challenge. This flies because there remains so much moral condemnation of drug users.
LikeLike
Allison
April 21, 2014 at 9:30 pm
Fabio – I also meant my questions literally. I’m curious about the methodologies of these studies. They say they did experiments right? So what were the control groups?
LikeLike
brayden king
April 21, 2014 at 10:08 pm
Most of the studies that have examined AA success rates are extremely limited methodologically. Large scale longitudinal studies are pretty much nonexistant due to the anonymous nature of the program. The studies that do exist suffer from heterogeneity in implementation and selection bias. Arguably the best experimental study, Project Match, assigned participants to one of three treatments. There was not a proper control group. All in all, the evidence is extremely murky but it seems like the best treatment option out there.
With respect to the cost, I think it is important to distinguish treatment centers from AA itself. Yes, treatment centers (which are in no way affiliated with AA) can be ungodly expensive. However, the AA preamble explicitly states that “there are no dues or fees for AA membership.” Most people contribute $1-2 a meeting to cover expenses. Viewed in this light, even a 5-10% success rate for ~$365 a year looks a lot different from the argument presented above.
LikeLike
Anonymous
April 21, 2014 at 11:49 pm
A lot of good discussion. A few notes:
1. Allison – great comment – this sheds a lot of light on this medical institution and why it is hard to study with the tools that we prefer (RCT or longitudinal observation). But still, if no one can proffer strong evidence of ANY type, then the presumption should be skeptical: “This might work and it’s worth figuring out, but until we do, we should exercise extreme caution when we encourage people to do it.”
2. Anonymous – Once again, it’s an issue of relative cost. AA may be popular because it has a non-zero effect (maybe) and has a very low cost. From a policy perspective, it may be justified given that the innovation seems to be on the margin (e.g., as Allison said most treatment is a variant of a basic model). But once you switch to residential treatment, the cost-benefit abruptly switches and we need more to counter the presumption of skepticism.
LikeLike
fabiorojas
April 22, 2014 at 12:27 am
Fabio, when you say, But still, if no one can proffer strong evidence of ANY type, then the presumption should be skeptical: “This might work and it’s worth figuring out, but until we do, we should exercise extreme caution when we encourage people to do it.” I am left with the strong sense you don’t actually understand the medical consequences of addictions such as alcohol. These conditions are reasonably often ultimately fatal. Even before the point of fatality, they can maim, dement, and disable; in the throws of the addiction, the addict may disorganize his life so profoundly he may destroy his career, his family, his property, and his legal standing as a non-felon. Addictions can make the addict a danger to others, whether through disinhibiting rage, or impaired judgment (e.g. drunk driving, child neglect), or increased risk of transmission of infectious illness (e.g. Hep C, HIV).
You do not seem to appreciate the profound seriousness of addiction in a medical sense. The risk of doing nothing is not negligible. And rates of recovery do not capture the sometimes permanent human harm that the addict, their family, or the community a large may be left with even if the addict never uses their intoxicant again.
I do not at all disagree with your gist. The evidence for AA/NA is terrible. But “exercising extreme caution when we encourage people to do it” makes no sense when 12 step meetings are functionally free, and the consequences of doing nothing are often so dire.
The fundamental reason AA/NA persist is not that they’re some sort of racket which has seduced the marketplace of patients away from better treatments. It’s because there’s basically no alternative. If you have one up your sleeve, the world will beat a path to your door for it.
LikeLike
Minder
April 22, 2014 at 6:54 am
Some parts of this discussion has brought to my mind that the famous statement of Zucker (1983): “institutionalization simply constructs the way things are: alternatives may be literally unthinkable” (p. 5).
LikeLike
Umut Koc
April 22, 2014 at 9:56 am
Fabio, I don’t really understand your comments. You do realize that many tough alcoholics certainly have no mortgate to pay any rehab, since it’s all gone into the alcohol addiction and simultaenous loss of a job? As Minder says, I do not get the impression that you in any way realize how severe alcoholism can be. These are not people that lead regular lives, but they will stumble around drunk in full daylight, not being able to hold a job, and thus robbing children of any functional parent. AA meetings are as you acknowledge yourself almost free – how can one in any way try to dismiss this as a viable option, and something one shouldn’t do? I simply don’t understand it. Doing nothing compared to doing something that could help – it’s not a question about the risk of non-success; it’s a question of the risk of failing “because” one didn’t do anything at all. Many children of alcoholics blame themselves for not doing enough, and that is a severe pain. Not really measurable of course, so probably not of interest to policy makers.
LikeLike
The first Anonymous
April 22, 2014 at 1:30 pm
Given the rhetoric of AA (and other 12 step programs), I’m not sure there is any empirical evidence that could dissuade advocates of their system. If (from their perspective) one is always an alcoholic–forever and ever–then there is always a need for AA type services. I’m not sure they would even take pride in the few “successes” of their program given that one can never be fully “cured,” former alcoholics are still alcoholics (just in a state of remission until their next relapse).
Places like AA do the moral dirty work of society. They do work that we (outsiders) want done, but don’t want to do ourselves. As long as that is the case I don’t see them going away anytime soon–no matter how much empirical evidence there is of their ineffectiveness.
LikeLike
kenkolb
April 22, 2014 at 1:31 pm
Just to clarify. I’m not sure I agree with the idea that alcoholism can’t be cured. I’m just paraphrasing their position that once an alcoholic always an alcoholic.
LikeLike
kenkolb
April 22, 2014 at 3:46 pm
Perhaps it would be more fruitful to compare programs like AA with religious institutions, rather than medical institutions? Some have already pointed to the cult-like quality of these programs, and that seems to help us understand why they endure even though there is a pretty low rate of success, if we call success getting and staying clean (which is problematic for the many reasons that people have pointed out). But the observation that these programs are cultish should direct us to consider that they do more that address addiction in a medical sense. They are arenas of moral integration formed in and through a set of rules. What people end up bonding over may not be the upholding of the rules, but how the rules of conduct within the occasion facilitate a sense of solidarity over the difficulty of attaining to the moral standard of relatives, friends, employers, etc. If you have ever been to an AA meeting, you might have seen how people build relationships through relating accounts of moral failures in other parts of life. What efficacy means is up for debate here, and this seems to stem from the character of addiction as a problem. The ‘cure’ for addiction is not the same as other medical interventions. One does not need to engage in a process of reforming internalized morals in order to get cancer treatment (okay, I could see how this might happen in certain cases, but it isn’t a necessary condition). Addiction is primarily a social problem, and secondarily a physical problem.
LikeLike
art
April 22, 2014 at 3:53 pm
“We should probably get tougher on organizations that claim to do good”
I’ll get right on it.
LikeLike
Rich
April 22, 2014 at 8:15 pm
@kenkolb: “Given the rhetoric of AA (and other 12 step programs), I’m not sure there is any empirical evidence that could dissuade advocates of their system. If (from their perspective) one is always an alcoholic–forever and ever–then there is always a need for AA type services.”
Actually, I don’t believe that logically follows. AA (about which I know more than other 12 step orgs) does hold, as I understand it, that once an alcoholic, always an alcoholic, true, but AA does not hold that the condition is untreatable, merely incurable. It does not hold that one will forever need to be in treatment, merely that recovery requires that one never drink. AA is very much based around a recovery model, where the goal is that the alcoholic attains a remission of his symptoms, and AA’s process is predicated on the idea that the addict must recognize and accept that, as an addict, he will require a certain behavioral discipline (not drinking, avoiding the stimuli he has identified will trigger his cravings, etc.) to remain symptom free and manage his illness.
Full disclosure: I’m a psychotherapist and I reasonably regularly treat cases AA and NA failed with. I am reasonably familiar with them through my patients, but I do not use the 12 step approach. There is minimal conflict between my approach and 12 step, because I believe their model of the illness is fundamentally correct, about which I’m happy to explain more if it’s of use to the discussion.
LikeLike
Minder
April 23, 2014 at 2:31 am
@art: “One does not need to engage in a process of reforming internalized morals in order to get cancer treatment (okay, I could see how this might happen in certain cases, but it isn’t a necessary condition).”
Cancer? No*, but much of the challenge described of treating addicts turns out to be true of treating other conditions through behavioral change. For instance, managing Type II Diabetes or heart disease through diet, exercise, and stress management: it can be done, but I understand treatment compliance rates for these things are pretty dire, and have relapse profiles pretty similar to those of addicts in treatment. If anybody actually has the analogous numbers for those things, it would be keen to see them.
Getting people to change their behavior is hard. Even when you put a barely metaphorical gun to their heads and say, “You must never eat fried chicken again, or you will die.”
* Don’t quote me on this, as I am no authority on it, but I’m given to understand that the commonest cause of lung cancer is tobacco consumption, the commonest cause of throat cancer is alcohol consumption, and the commonest cause of liver cancer is viral infection transmitted by sharing needles used for IV drug abuse. So.
LikeLike
Minder
April 23, 2014 at 2:50 am
(Oh, dear. I don’t seem to be able to manage my HTML today. Please feel free to delete the above. Sorry about that.)
@art: “One does not need to engage in a process of reforming internalized morals in order to get cancer treatment (okay, I could see how this might happen in certain cases, but it isn’t a necessary condition).”
Cancer? No*, but much of the challenge described of treating addicts turns out to be true of treating other conditions through behavioral change. For instance, managing Type II Diabetes or heart disease through diet, exercise, and stress management: it can be done, but I understand treatment compliance rates for these things are pretty dire, and have relapse profiles pretty similar to those of addicts in treatment. If anybody actually has the analogous numbers for those things, it would be keen to see them.
Getting people to change their behavior is hard. Even when you put a barely metaphorical gun to their heads and say, “You must never eat fried chicken again, or you will die.”
* Don’t quote me on this, as I am no authority on it, but I’m given to understand that the commonest cause of lung cancer is tobacco consumption, the commonest cause of throat cancer is alcohol consumption, and the commonest cause of liver cancer is viral infection transmitted by sharing needles used for IV drug abuse. So.
LikeLike
Minder
April 23, 2014 at 2:52 am
erm… also the article is written by a psychiatry MD. Yes, I know, we should all get meds for every addiction, but a 5-10% ‘success’ rate without getting addicted to antipsychotics is a chance I’m willing to take if I were to become an alcoholic
LikeLike
sd
April 24, 2014 at 5:59 am
Many of you are suffering from action bias. You can read about it here: http://ideas.repec.org/p/pra/mprapa/4477.html
More studies= more cost and will not ameliorate the fact that we have policies that are actually worse than doing nothing.
LikeLike
August
April 24, 2014 at 3:58 pm