orgtheory.net

financializing social services

Do people know about social impact bonds? I hadn’t heard of them till recently. Since then, though, I’ve developed a train-wreck fascination. They have the potential to combine all the worst features of the public and private sectors. And they can be securitized, to boot!

Let’s take a step back. What is a social impact bond, anyway?

Well. Imagine you have a social problem you’d like to solve. Say that you want to reduce recidivism among young people in prison. That sounds good, right? The problem, of course, is that taxpayers don’t want to pay for rehabilitative programs, and there’s lots of disagreement about what kind of program would actually help solve the problem, anyway.

The government says, Wouldn’t it be nice if somebody would take care of this for us, and we’d only have to pay them if they actually succeeded?

Enter Goldman Sachs.

To oversimplify a bit, Goldman Sachs (or some other investor) puts up the money to implement the program for youth in prison. (This is a real example.) They provide the money to a social service agency that actually carries out the program. There’s a target that indicates success—let’s say, cutting recidivism by 10%. If the target is hit, Goldman gets paid back—with interest. 22% in this case, and Bloomberg Philanthropies is being nice enough to cover most of the potential loss. Yay, profit! But if the target isn’t met, Goldman doesn’t get paid back in full. Boo, losing money.

Sounds good, right? Goldman takes the risk, everyone has incentives to succeed, and taxpayers only have to pay for successful programs. What could possibly go wrong?

Well, I can think of a couple of things.

First, social program success is difficult to measure and very game-able. When an investor has a large interest in hitting some narrowly defined “success” target and little or no inherent interest in the actual, long-term outcome, there are going to be efforts to game the system.

Think of law school deans trying to beat the rankings in ways both frank and fraudulent—and one assumes that they are not only in it for the money.

Or think privately funded charter schools where the investors only get payouts if test scores hit a certain target.

Or, closer to (Goldman’s) home, think ratings agencies who will lose important clients if they don’t come up with ways to produce favorable ratings on financial instruments.

Investors whose payout depends on hitting a number + social service agency whose revenues depend on demonstrating “success” = lots of potential for bad behavior.

Second, if social impact bonds catch on in any significant degree, they are going to end up creating all sorts of new groups who have a big interest in creating bonds and only a secondary interest in long-term social outcomes. The folks promoting these should be reading Josh Pacewicz’s excellent work on tax increment financing.

Tax increment financing is a now-common way of financing municipal economic development projects. It allows cities to sell bonds secured by the future increases in tax revenue that are expected to result from a development project. So the city borrows money for, say, waterfront development, and promises to pay it back with the tax revenues that it projects will come from the newly developed waterfront. But if the revenues don’t materialize, the city isn’t on the hook. Like social impact bonds, part of the appeal is that the risk is shifted away from the taxpayer.

The problem, though, is that cities have gone overboard with TIF. This means high debt levels and committing most of the tax revenues resulting from development to paying back the debt that made it possible.

According to Pacewicz, TIF didn’t take off because of huge investor demand for the debt. Its expansion was driven by a new group of economic development professionals who worked for cities. TIF gave these professionals influence because it made municipal finances more complex—to the point where they gained control over city budgets because no one else understood them.

These professionals are very aware that TIF is overused—that projections of tax revenue increases are too optimistic, that cities are taking on too much debt, and that many development projects are basically corporate welfare. And they regret it. But they advance in their careers by putting together big TIF projects. So they put together TIF projects, even when they think it’s bad for the city.

How does this apply to social impact bonds? Well, if SIBs take off, we can expect to see a new group of government professionals emerge whose job it is to create social impact bonds. And they, too, will most likely want to advance their careers by creating big, splashy SIBs. This could lead to the replacement of plain-vanilla government-provided social services with privately financed services, even if the latter cost more or involve more layers of bureaucracy than the original ones.

Finally, if you’re not convinced that social impact bonds will go very wrong in either of those ways, here’s the kicker. They can be securitized! Massachusetts is structuring its social impact bonds—also targeted at recidivism—like collateralized debt obligations, to expand the pool of potential investors by creating lower-risk tranches. What could possibly go wrong?

The killer is that this is really an initiative coming from the left. It’s Rockefeller-supported and championed by the Center for American Progress.

I have no doubt that many of the people involved—maybe even at Goldman—are genuinely and deeply motivated by a desire to find innovative solutions to social problems. (Okay, maybe not at Goldman.)

But this kind of solution reflects way too much faith in our ability to distinguish between successful and unsuccessful programs—which is absolutely key to the whole thing working. And it involves way too little cynicism about how hard it will be to create a structure that works the way it was intended, and does not simply become self-perpetuating, regardless of cost or degree of success, by creating lots of people who have an interest in keeping the thing rolling along.

It’s possible social impact bonds won’t go anywhere. One of the earliest recidivism experiments in the UK just announced that investors won’t be paid back yet, though they’re on track to be paid in two more years. And so far most of the actual support for SIBs has come from philanthropists, not private investors.

Nevertheless, SIBs have the flavor of the next big thing. They promise to encourage private capital to support innovation in publicly supported social services. What could be more broadly appealing?

If only people’s enthusiasm for evidence-based policy extended to policy innovations designed to promote evidence-based policy.

About these ads

Written by epopp

August 11, 2014 at 1:01 pm

9 Responses

Subscribe to comments with RSS.

  1. Those concerns sound more like scope conditions than actual criticisms of SIBs (Social Impact Bonds). Measurement in the social sector is often hard, but in some areas there’s great measurement and widely agreed upon metrics. Contracts of any kind can be gamed… but, there’s a lot of smart people out there who research ways of designing contracts that limit gaming. Moreover, I think it’s pretty clear that SIBs won’t be a panacea for funding public services… but very few advocates make such claims.

    Recidivism is a great example of a good metric – it’s already collected as part of the criminal justice system and is widely agreed to be a meaningful measure. It’s also really hard to game – though not impossible – to game. I suppose that a really crafty organization could pay for lawyers to help offenders dodge prison but that seems like a stretch.

    Personally, I think that sociologists actually have a lot to add to the conversation on defining these scope conditions. Issues like the social construction of measurement and commensuration are salient here and should be part of the SIB conversation. Understanding how and why organizations develop and institutionalize performance metrics is definitely in our wheelhouse.

    And we should be wary of metrics that are pushed by organizations. The example of microfinance comes to mind here; you have a field that institutionalized a set of metrics, largely based around repayment rates, that were not correlated with the outcomes of interest; namely poverty reduction.

    I think your claim that “this kind of solution reflects way too much faith in our ability to distinguish between successful and unsuccessful programs” is not fair to the designers of the SIBs. In the contracts I’ve seen, they’ve taken great care to make sure reasonable counterfactuals are established – including random assignment to the SIB program with reasonable sized Ns (in the hundreds) for each group. That’s a higher standard than most sociological or management research out there. Note that these control groups are above and beyond the existing evaluations of the providers, which tend to be pretty well executed too.

    Also, I’m not sure exactly why these apparatuses would become anymore self-perpetuating than the existing nonprofit infrastructure. If anything, investors may have some incentive to support organizations that are struggling to meet their targets.

    Finally, this movement is bigger than SIBs. The larger movement is a push towards “Pay for Performance” – funders paying for outputs of nonprofits (e.g., job placement) rather than inputs into nonprofits (e.g., job training). A bond is just one way of financing this but one could imagine a nonprofit financing its with their own capital. Or a loan.

    Liked by 1 person

    jmodel

    August 11, 2014 at 11:11 pm

  2. Great post and a good first reply. Lots of stuff to chew on. But I’m expert enough to pick one nit in jmodel’s reply re this:
    “Recidivism is a great example of a good metric – it’s already collected as part of the criminal justice system and is widely agreed to be a meaningful measure. It’s also really hard to game – though not impossible – to game. I suppose that a really crafty organization could pay for lawyers to help offenders dodge prison but that seems like a stretch. ”
    This quotation signals to me either a lack of knowledge of the CJ system or a person who has totally bought into a CJ profession’s mindset. Because “recidivism” as it is always measured is the behavior of the CJ system and only incidentally or at best partially a measure of the behavior of the offender. First, varying by state, a majority of “recidivists” in the sense of being returned to prison are returned for violations of the terms of supervision, not for crimes, and there is usually a great deal of discretion given to probation/parole officers about how to handle such violations. Second, even if recidivism is measured as commission of a new crime, the measure includes not only criminals behavior, but the behavior of the CJ system in monitoring and catching the criminal behavior, not to mention the discretion CJ professionals have in how to respond to low-level crime.

    Liked by 1 person

    olderwoman

    August 12, 2014 at 12:07 am

  3. PS there are many studies that show that people receive wildly different penalties for the same act depending on their resources and lawyering. Just to emphasize the points of discretion and potential bias there are: monitoring and policing practices that affect whether police ever see the act; police discretion in how to respond to the act; prosecutor discretion in what crime to charge the person with (any given act typically can be construed as a violation several different laws that carry very different penalties; the plea bargaining process involving a defense attorney (if any; no free lawyers if you are not destitute and not charged with a serious felony) and a DA (at least 97% of criminal cases are adjudicated by plea bargain), and the judge’s sentence. Then after sentencing, if you go straight to probation, are are subject to probation officer’s discretion about violations. If you go to jail or prison, you are subjective to administrative controls that will affect your behavior inside and your chances of getting out. Then you are subject to more CJ discretion in the community supervision.

    Like

    olderwoman

    August 12, 2014 at 12:13 am

  4. @olderwoman Just to respond. First – full disclosure – I’ve worked in the criminal justice system and familiar with the idiosyncrasies of measurement and discretion.

    However, I think where we disagree is what recidivism is measuring. For the reentry SIBs, it’s really about about outcomes for the state – namely keeping people out of jail or prison. Does it directly measure crime rates like victimization surveys do? No. Does it measure income or employment? No. But the framework could accommodate those measures if someone wanted to. And I think recidivism is a decent measure in that it does capture some social benefit for taxpayers (reduced spending), the potential recidivist (not going to jail or prison) and the potential recidivists’ family/friends etc.

    Moreover, since there are control groups in a similar geography, I think there’s less concern regarding jurisdictional heterogeneity. The SIBs are structured to pay for success above and beyond the control group.

    So you would have to believe that the CJ system would somehow differentially enforce parole or crimes between the control and treatment group for this to be a real concern. That’s possible; in fact, my first post gave an example of something that a nonprofit could do to game the system by providing additional legal resources. Perhaps the police and courts know who is in the SIB treatment group and are more lenient or harsh on them? To some degree, that’s empirically testable through court records.

    But I think the broader point I was trying to make was that this is exactly the sort of conversation I think sociologists could be engaging in with the SIB proponents. Any summative metric, like recidivism, is going to be flawed in some ways. So understanding the failing or even documenting organizational misconduct in gaming measures should hopefully allow for a more reasoned approach. These are the types of things that qualitative and quantitative researchers are really good at!

    Like

    jmodel

    August 12, 2014 at 12:36 am

  5. jmodel, first to reiterate that I thought the thrust of both the OP and your comment were worth thinking about. I just reacted to the idea that recidivism couldn’t be gamed as a measure. Because I can think of a lot of ways to game a recidivism measure, depending on whether my goals was to make it go up or go down. I was just reacting (overreacting) because there are all these studies of recidivism trying to ask which in-prison treatments work best, blah blah, but they are written as if the only problem is to get offenders to do less crime, and none of the problem is to change the behavior of the people creating recidivism.

    But to your larger point, that dialog needs to happen, I agree. I keep trying to insist* that racial disparities ought to be measured and monitored because that is the only way way to get CJ actors to care about reducing disparities and have spent a lot of time on this very specific issue of using a measure to try to hold agencies accountable. But you have to target the “program” on the right actors.

    * I’m not the only one insisting, of course, there’s a whole group of reformers who care about this, I’m a relatively small fish, but I have spent a lot of time on the issue in my area.

    Like

    olderwoman

    August 12, 2014 at 1:56 am

  6. Thanks for the thoughtful comments.

    @jmodel, I do appreciate what you’re saying here, and certainly I would rather see sociologically informed informed discussion of good/bad metrics than no such discussion. (This article — http://s3.amazonaws.com/mildredwarner.org/attachments/000/000/413/original/d116aef904347207d0dd8d5916616045 — written by a scholar with a sociology PhD, was the best effort I saw at addressing this very new policy innovation from a sociological perspective).

    You see my criticism as really reflecting the need to identify scope conditions under which SIBs would work well or poorly, so that we can choose when they might be an effective solution and when they won’t work because they can be gamed, or the metric doesn’t align with the actual desired outcome, etc. I think where we might fundamentally differ is that while I believe it’s possible to identify such scope conditions in the abstract — as academics, we can identify and describe them — politics means we will never be able to take that knowledge and effectively implement it. Instead, politics (both big-“P” and little-“p”) will always get in the way and keep the implementation from matching the ideal.

    Because of that, I have a strong preference for simplicity in policy design. The more complex the mechanism — and this is a pretty complex mechanism, with several layers of organizational intermediaries, outside evaluators, etc. — the less likely I think it is going to be implemented in a way that even vaguely approximates the abstract ideal. Under these circumstances, all the potential negative effects, like gaming, bias in what kinds of indicators are selected, unintended consequences, etc., are more likely to show up.

    Some of my strong negative gut reaction to SIBs also comes from studying efforts to use social science to inform policy over the last 50 years. I think of the negative income tax debates of the 1970s, for example, and how the effort to test the effects of NIT experimentally was almost immediately derailed by political demands for premature reporting of results, politicians pushing to interfere with experiments in various ways, endless debating over how to interpret the effects, etc. Or, even earlier, efforts to rationalize budgeting in the executive branch by applying systems analysis in an attempt to identify the most cost-effective ways to meet predetermined goals, which turned out to be totally manipulable by both bureaucrats and politicians.

    I’m certainly not averse to social-scientifically-informed policy innovations. To pick a random example, I think some of the simpler suggestions of behavioral economics (e.g. opt-out v. opt-in retirement savings) make a lot of practical sense. But I think social scientists have a long track record of overoptimism regarding how easy it will be to implement policy ideas in ways that even vaguely resemble the original ideal. And I guess it’s that, fundamentally, that makes me so pessimistic about SIBs.

    Like

    epopp

    August 12, 2014 at 7:23 pm

  7. @epopp

    So first, I appreciate the link and the discussion – I’m actually co-teaching part of a course that includes a discussion of SIBs in the fall – so this is really helpful stuff.

    The point on the distance between academic distinctions and implementation is well taken. But I actually don’t think SIBs are that complicated when they are boiled down. SIBs really consistent of two things that have been around for a while:

    1) A performance contract between a service provider and a funder
    2) Some source of capital – in this case a bond.

    Regarding #1, performance contracting is really nothing new. For instance, lawyers often work on a contingency basis. Tipping, in theory, is a form of a performance contract. You pay after you’ve received service. Pretty much any contract where one works on commission is a form of this. So this isn’t especially novel in the for-profit – or even government – world. In the nonprofit sector, for whatever reason, performance contracting is often eschewed. Part of this seems idiosyncratic – development officers don’t work on commission although organizations that do fundraisers often do – and part of it is because it is hard to measure outcomes.

    But there’s been a lot of progress in building infrastructure in the sector in order to measure outcomes in systematic ways. Randomization is a pretty good way to make apples-to-apples comparison. It’s certainly not foolproof – indeed, one could imagine people working hard in order to make a randomization process not actually random through creaming of various sorts – nor is it costless. Nevertheless, applying these techniques that are already pretty widely used in the sector to make performance contracts doesn’t seem like a huge leap.

    But performance contracts have the issue that one gets paid after a service is rendered. Maybe this isn’t a big deal to the restaurant server who only has to wait an hour for a table to clear, but it is clearly an issue for a nonprofit that may have to wait several years and incur the cost of their programs in the meanwhile. If the nonprofit sector were well capitalized, this would also not be an issue; a nonprofit could take on the costs and get paid once the services were rendered. However, nonprofits aren’t well capitalized enough to deficit spend like this. They need to have someone to allow them to take on debt.

    In fact, the B-part of the SIBs is the least interesting part to me. Bonds are just one instrument of debt – one that nonprofits occasionally use (often for capital expenses like construction). One could just as easily talk about SILs if the L were for loans. Perhaps that would make things seem less complicated?

    See links for examples of nonprofit bonds:

    http://harvardmagazine.com/2011/01/back-to-the-bond-market

    http://www.orrick.com/Events-and-Publications/Documents/172.pdf

    http://www.sandiego.gov/economic-development/business/financing/nonprofitbonds.shtml

    And I think the greatest promise in SIBs or performance contracting in the sector lies in actually having a well capitalized nonprofit sector. Specifically, I could see shifting foundation assets from hedge funds to being used to back SIBs or performance contracts. Or perhaps in addition to donating money, individuals would provide capital to back these contracts. Actually, this is already happening in the nonprofit sector – it’s basically how Kiva works! (see http://www.kiva.org/about/how/more )

    In sum, I think that SIBs are not inherently complex – especially when they are decomposed even just a little. I would agree that the initial contracts are clearly complex their length and detail, but I would attribute that to their age rather than the concept as a whole. Perhaps the obvious point missing from this discussion is that there are surely other ways to solve the primary problem – the lack of capital in the nonprofit sector – besides issuing debt. But I think that debt instruments, be they SIBs or something else, should be one tool in the toolbox.

    Like

    jmodel

    August 13, 2014 at 1:17 am

  8. @jmodel, great comment. So when I argue against complexity, I’m not referring to the bond part so much as the multiple intermediaries involved in trying to accomplish the government agency’s objective. This MDRC illustration (http://www.mdrc.org/key-partners-nycs-social-impact-bond), for example, shows 1) foundation supporters, 2) private investors, 3) MDRC as “intermediary,” 4) the program operator, 5) government contractor, and 6) outside evaluator. That’s a lot of parties that have to be brought together around the question of what an appropriate target is and how we will know if it is reached. I agree with you that there is probably some range of circumstances in which one would be trying to achieve a well-defined outcome that is straightforward to measure, difficult to game, and that nicely aligns what is measured with the larger social goal we want to achieve. But I think such circumstances are likely pretty rare. Obviously recidivism is one where people must think that’s the case, since that’s where a lot of the action is; I don’t know enough about recidivism to have an opinion on that specifically, though OW raised some possible issues above.

    But evaluating program success is, in general, really, really hard. I think of Peter Rossi’s Iron Law of Evaluation (The Iron law: “The expected value of any net impact assessment of any large scale social program is zero. The stainless steel law: “The better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero.” http://www.gwern.net/docs/1987-rossi).

    My concern is that innovations like this have a tendency to become bandwagons — the hot new solution — and then get applied all sorts of places where they’re not really a good fit, but some government administrator decides that they need some SIBs to look like they’re up to date with the latest financing mechanisms, and some nonprofit sees an opportunity to expand, and some investor sees that Bloomberg will cover the risk and they can make some money. That’s where I think there’s lots of opportunity for creating a system in which government pays more than the cost of social services without necessarily getting higher quality/more successful services than they would have under other circumstances (because I’m imagining it happening in difficult-to-evaluate or easy-to-game situations), and it could have the tendency to become self-perpetuating.

    I guess there are several possibilities here. One is that people promote the idea, but it doesn’t take off because targets are hard to achieve and private investors don’t want to invest. A second is that these are used in specific circumstances with well-defined targets. A third is that they become trendy and are used a lot but largely in situations where you can’t easily define a good outcome and/or outcomes are manipulable. You think we can hit #2. I think #1 or #3 are more likely.

    Finally, as you also mention, to the extent that this is about solving the problem of lack of capitalization in the nonprofit sector, there is an underlying issue here that can be forgotten in the details of the particular financing mechanism.

    Anyway, I do appreciate your measured and very well-informed response.

    Like

    epopp

    August 13, 2014 at 9:13 pm

  9. I would caution against drawing general conclusions about cities and TIF from Pacewicz’s work. Perhaps most important is the fact that states play a major role in what cities are able to do with respect to economic development–thus some cities use TIF only sparingly because of restrictions from their state government. This is, in part, why TIF is everywhere in Chicago and nowhere in Seattle. Second, some city finance departments are risk-averse, thus less likely to permit the use of TIF even if a few economic development professionals want to use it for every development project. Many major cities–perhaps outside of the two anonymous Iowa cities Pacewicz studied–contain highly trained finance officers that are less likely to be confused or bewildered by the financing products economic development professionals propose.

    Of course, state and municipal variation in using TIF is not Pacewicz’s project. Wouldn’t want to critique his work based on questions he doesn’t claim to answer. But it appears that since he’s been the first to publish (in sociology) about the current financing tools of economic development in cities, folks seem to think these processes are more universal than they are. And I would caution against that.

    Like

    J

    August 27, 2014 at 1:13 pm


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 1,184 other followers

%d bloggers like this: