Skip to main content
Sep 20, 2013

Social Impact Bonds and the Dilemma of Measurement

Ohio has been chosen as a trial state for development of social impact bonds (SIBs) under the programmatic title Pay for Success.  This new investment vehicle comes from the United Kingdom and has recently been inaugurated in a transaction between the City of New York and Goldman Sachs.

The central characteristic of a SIB is that the return to the investor is not based on an interest rate but rather on the degree of achievement of a specified measurement of impact.  Without going into the purpose and merits of SIBs, this investment vehicle warrants a deeper discussion of the concept of measurement of social impact.

While measurement sounds like an objective concept, measurement of social impact is not.  Measurement in the sciences, psychology, and economics, to name a few, is constantly being re-evaluated and refined in sophistication and complexity.  With a head start of decades in these fields, advocates of measurement of social impact should be cautious and flexible, recognizing that its beginnings will be crude and inaccurate, if accuracy can  be achievable.

How will holders of SIBs respond to social impact measurements that will determine the financial return they receive?

As with so many new concepts, it is appropriate to remind ourselves of the challenges to good measurement programs.  The tendency is to measure what can be measured when the focus must remain on what should be measured.  There is also a tendency to measure items which already have good measurement techniques available.  This can lead to good measures of secondary items because of discomfort with using imperfect measures of what may be more central to the success of the undertaking.

To resolve these two challenges, it is critical to revisit why we want measurements in the first place.  I suggest the purpose should be to assess progress, not to judge achievement or to determine if an end point has been reached.  Specifically, I believe measurement is valuable for three reasons:
•    To provide a means to place focus on relevant issues.  I have been at too many board meetings where the focus is on operational details (easier to measure) rather than on strategy and mission (harder to measure).
•    To drive change and track progress.  Implementing change is a fragile process and a sense of progress or a recurrent reminder of the need for change is essential in order to maintain momentum and counter emergent worries that can eat away at support and enthusiasm.
•    To prompt questioning of current approaches and procedures.  In my experience data can be only as good as the questions asked.  Beginning a program of measurement of relevant issues can begin a beneficial cycle of more insightful questions leading to better data and in turn leading to better questions.

A danger of a measurement program can be to turn from these three purposes and instead make measurement a means of judging the skills and effectiveness of people.  In other areas with more widely accepted and objective measures, this may work, but not for social impact.

Only after reaching a clear agreement on purpose, can one confront the challenge of accepting imperfect measures of social impact.  Measuring social impact is a messy, evolutionary undertaking.  The method and the measure should be reviewed and possibly revised each year as one finds better ways to address the reasons for having a program of social impact measurement.

Social impact bonds are a thought-provoking new way for government to fund social programs.  I just hope their need for precise measurement today doesn’t draw attention away from the more important reasons for measuring social impact.