Can you, or indeed should you, put a price on philanthropy? After all the money is being spent to benefit humanity, so isn’t it morally suspect to apply such base criteria as value for money?
An alternative view is that checking that the money has been spent well ensures that, to put it crudely, you get ‘more bang for your buck’ and the investment goes further.
We commissioned a report on the ‘Social Return on Investment’ SROI) of our Young Health Programme in 2017, with exactly this in mind. Fortunately the results came back positive, but it was the journey getting to those numbers that really helped us to understand how our programmes actually delivered value.
Our multi-million dollar investment in the Young Health Programme (YHP) ranges widely – from research to advocacy to on-the-ground programmes; from country to country; and in scale from small activities involving a few dozen people to those benefitting hundreds of thousands. They all share the same objective though, to improve young people’s health through behaviour change – and especially to reduce the likelihood that they will contract non-communicable diseases (NCDs) such as heart and respiratory diseases, cancers, and diabetes.
In the process of examining the options it became clear that it would be very difficult to attribute a particular value to the research and advocacy elements of the YHP, as there were likely to be a lot of other factors involved. Was it our communication with a minister that changed the policy, or was it our side event at the UN General Assembly?
For small-scale behaviour change programmes, and especially those with hard-to-reach individuals and communities, any form of data collection was very challenging. This ruled out getting meaningful results. Our relatively large-scale structured programmes were quite a different matter. Here we had programmes of some duration, involving many people, and with more reliable data. We decided to focus the analysis on the programmes run with our partners Plan International in Brazil and India; with the Boys and Girls Clubs of Canada in Canada; and with Junior Achievement in Romania.
These were all aimed at changing the health behaviours of adolescents – although the focus varied form country to country: general mental wellbeing in Canada; encouraging healthy diet and physical activity in Romania; and a broader approach to reducing ‘risk behaviours’, including an emphasis on gender equality, in Brazil and India.
Identifying the financial investment, or the money put in to support the programmes, was relatively simple. Defining the ‘social return’ was rather more complicated. The ultimate aim of all of the activities was to reduce the level of premature death (mortality) and illness (morbidity) that these young people were going to experience in later life.
This requires a couple of leaps. Firstly, what would have happened to them in the future if they had never been involved in the programme? Secondly, what difference was their involvement in the programme likely to have made to this outcome? Thirdly, how could a financial ‘social’ value be attributed to this difference? This was made especially difficult when in Brazil and India we were working with particularly disadvantaged groups, who would may be more likely to be exposed to problematic lifestyles and behaviours than the average, and so differ from the national statistics.
Identifying how many young people are likely to take up and maintain risk behaviours such as poor diet, smoking or alcohol abuse into later life is relatively simple as there are national estimates of the current situation collated in the Global Burden of Disease (GBD) survey 2015. There is of course a time lag in these data. For example if we are concerned that a teenager may be obese and suffering from diabetes in his mid fifties we don’t know what the incidence of obesity or diabetes will be then. Will it have increased, as present trends suggest or, like smoking in many countries, have declined?
The effect of the programme itself on behaviour can also be a complex calculation. For some programmes there is self-reported evidence of behaviours at the start, middle and end of the programme. The Brazil programme reported that “There was a reduction in the number of young respondents who reported using substances down to 13% in the end-line, compared to nearly 50% in the mid-line evaluation”. This seems to indicate a very high level of effectiveness, but to what extent were people in this age group likely to stop using substances anyway? How many of them are likely to revert to their old ways after they leave the programme and what proportion will give up anyway as they enter adulthood?
To quote Donald Rumsfeld, the retired US politician, these are ‘known unknowns’ or things that we are aware that we don’t know; so for this we had to build in assumptions based on other evidence, common sense, and always on a conservative view of likely effect. So in this example the researchers assumed that 19 out of 20 people (95%) who attended the programmes and reported not having used the substances would then revert in later life. Obviously we hope that the real number is much lower but we did not want to make the results of the programme unrealistically positive.
With this basic understanding of the effect of the programme we had to work out how many people had been affected and how intensely. The programmes all used, to varying extents, ‘peer-educators’ and these young people had talked about the subject many times to their friends, family and others – and had to demonstrate their words with their actions – and so were far more likely to maintain a healthy lifestyle into adulthood than those on the very fringes of involvement. These various levels of interest and involvement had to be allowed for too.
From this point on the number of people likely to have permanently changed behaviour could be calculated and, using a key source (the huge Global Burden of Disease survey), converted into a financial estimate of how much cost had been avoided in such areas as treatment, lost productivity and reduced taxable income. Factors like this comprise the social value included in the calculation – although of course there is a wider emotional social value that is beyond the bounds of this kind of study.
The ratios for our four programmes finally came in at between 6.5 and 8.8; meaning that for every dollar spent on the programmes they returned between $6.50 and $8.80 in social value. These numbers should not be taken too literally – as we’ve seen above there are lots of assumptions and some of the data are less than robust. These are therefore indicators of value rather than precise calculations of it.
Interestingly a completely separate study (Prevention for a Healthier America: Investments In Disease Prevention Yield Significant Savings, Stronger Communities) by the Trust for America’s Health looked at the social value of addressing NCDs in the United States and came up with an SROI ratio of 6.2. At the very least two completely separate studies, using different sources and methodologies, producing such similar results does suggests that investing in behaviour change at an earlier age works financially as well as transforming the life prospects of the people it serves.
Whilst this information confirms the value of running the activities, its greater value in helping us to understand how our programmes work in practice, and so how we can make them even more effective.
The core to developing an SROI analysis is to understand the process by which the social value is generated, usually called the ‘Theory of Change’ (TOC). Our programmes all address how behaviours in adolescence can be modified to reduce the risks associated with non-communicable diseases (NCDs) especially heart and respiratory diseases, cancers and diabetes in later life. How they go about this reflects local concerns and priorities. As a result they were quite varied in their approach and focus, but at their heart had a TOC we summarised as:
- Young people are recruited locally and trained as ‘peer educators’ in awareness of harmful behaviours, and in communicating how these can be avoided to other young people
- The young people beneficially change their behaviour, resulting in reduced morbidity and premature mortality
- Other audiences associated with the programme also change behaviour (teachers, parents and others affected indirectly through social and other media) at varying levels
Within this there were many differences between the programmes that could affect their impact, for example whether they addressed a very specific issue – such as tackling overweight and obesity as in Romania, or mental health in Canada, or took a more general approach to mental and physical well-being as in Brazil and India.
As we worked through the process it was clear we could not measure everything, would not have robust data for every aspect of the programmes, and so would need to concentrate on a number of simpler metrics that are relatively easy to collect. Academic studies provided useful pointers to some ways forward.
It has long been known that smoking and alcohol abuse are closely related. One authoritative early study1 found that pack-a-day student smokers were more than three times as likely to drink alcohol and 10-30 times more likely to use other illicit drugs than non-smokers.
A Minnesota study2 demonstrated a wider effect of smoking on exercise and diet; showing that daily smokers were less likely to undertake physical activity or participate in team sport - and about twice as likely to eat three or more meals of ‘fast food’ per week than non-smokers.
So ensuring that smoking is reduced is centrally important to making progress on changing other risky behaviours, and is something we should consider when setting up each new programme. It also provides a relatively simple metric of the level and frequency of smoking, which is possible to collect consistently.
Smoking is probably the most studied risky behaviour and so there is plenty of external research and data to draw upon and, as it works as an indicator for other behaviours, measuring smoking can help to give an indication of overall effectiveness. So charting smoking behaviour may also help with the metrics, keeping the level of information gathering to a practical level. This seems a fruitful way forward.
But smoking itself often stems from other issues – for example it is associated with major depressive disorder3 suggesting that deeper social and psychological support can be needed to stop people smoking. All of which suggests that more comprehensive programmes that work to address multiple issues – starting with self-esteem and smoking – are likely to be the most effective.
These considerations do, however, have to be balanced against what is practical in the field. Are the structures and resources available to provide such a wide approach, is it practical?
Fortunately in our 31 on-the-ground programmes we have a rich testbed of approaches to improving the life prospects of young people, which we can compare and refine. The SROI is just one important step on the way to using this virtual laboratory to maximum effect.
 Torabi MR, Bailey WJ, Majd-Jabbari M., 1993, Cigarette smoking as a predictor of alcohol and other drug use by children and adolescents: evidence of the "gateway drug effect".J Sch Health. 1993 Sep;63(7):302-6.
 Larson N., S. M.-S., 2007, Are Diet and Physical Activity Patterns Related to Cigarette Smoking in Adolescents? Findings From Project EAT. Preventing Chronic Disease , 4 (3).
 Brown, R.A., Lewinsohn P,M., Seeleym J.R. et al, 1996, Cigarette Smoking, Major Depression, and Other Psychiatric Disorders among Adolescents, Journal of the American Academy of Child & Adolescent Psychiatry Volume 35, Issue 12, December 1996, Pages 1602-1610