Social impact analysis and measurement is the growing hot topic among professionals involved in civil society organisations, philanthropy and social finance across Europe. However it is important to ask the question, are we all living in a fantasy world where we think social purpose organisations will implement and use impact analysis to improve their work?
In this blog I will give some real life examples of organisations and individuals from across Europe engaging in social impact analysis on a practical level, to counter the view that this is all one big fantasy. These examples are drawn from working in partnership with members of the Social Impact Analysts Association (SIAA) over the past four months to run a series of partner meetings or SIAA Impact Group launches in Estonia, Bulgaria, Hungary, Ireland, UK and Spain. Across these countries, with their differing political, economic and social landscapes, there was a surprising level of engagement and consensus around three burning questions in this field.
The first question was, why should we do impact analysis in the first place?
At each event there was a sense that many of the attendees were already won over by the idea of doing social impact analysis, but that there is still a need to discuss the main drivers for organisational and individual engagement.
During a meeting at Philanthropy Ireland in May, it was claimed that NGOs in Ireland are primarily engaging with impact analysis in order to satisfy funders. A claim similarly held in the UK and backed up by a New Philanthropy Capital study ‘Making an Impact’ (2012) which found ‘52% of charities that have increased their measurement efforts say they did so to meet funders’ requirements’. However the research also suggested that the main benefit for charities was actually using the analysis to improve services.
At events in both Bulgaria and Hungary attendees heard case studies from Dr Andreas Rickert CEO of Phineo in Germany, an organisation carrying out independent research for donors based on analysing the work of NGOs. Andreas outlined Phineo’s due diligence process, including analysis of all organisational dimensions of a non-profit from internal management to monitoring systems. An evaluation of Phineo’s work showed that 90% of organisations who went through the analysis reported they could improve their work as a result.
How then do we ensure that impact analysis is used to improve services and increase social impact?
At the Ministry of Social Affairs in Estonia practical examples, such as the use of dashboards and IT systems to monitor and analyse the progress of an organisation, were discussed at length. Jaan Aps from the Estonian Social Enterprise Network explained that such dashboards can be used to integrate analysis into the everyday work of an organisation. Further examples were discussed by reference to a recent Estonian social impact evaluation handbook (Jaan Aps, 2012) which has provided much needed language and context specific guidance in this area.
Similarly, practical examples were shared at an event hosted by the Foundation for Development of Democratic Rights in Hungary. Jeremy Nicholls Chairman of SIAA and CEO of The SROI Network shared plans for the development of a social value budget by the FRC Group in the UK. Such a budget would be something that could sit alongside the financial budget in an organisation. It would be used by the management team to monitor, analyse, report and improve upon their work in collaboration with their board of trustees. Jeremy also made reference to the SIAA Principles Working Group’s mapping of existing principles of impact analysis and a SIAA Thought Paper providing a framework for ‘how an organisation can approach the practice of social impact analysis’.
Attendees and speakers in Hungary were very encouraged by such case studies, but were aware of the perception in Hungary that this type of analysis is too complex for the current political and economic context. In response to these concerns a number of speakers such as Eva Varga from NESsT explained that organisations can take baby steps, starting with a simple approach such as theory of change to go beyond just measuring outputs.
In Ireland it was emphasised that good work is already being done on a sector-specific level to analyse outcomes in a collective way. For example the Prevention and Early Intervention Network and Children’s Research Network for Ireland and Northern Ireland are some of the increasing number of organisations trying to facilitate sharing of data and analysis on a sector and system level to influence change. The message being that we need to stop constantly reinventing the wheel and look to how we can use shared measurement approaches and existing evidence and data to make better decisions and improve. Similarly in Bulgaria attendees were very interested to hear the mention of a UK programme Inspiring Impact and its blueprint for shared measurement case studies, such as the outcomes measurement framework Insights from the CAADA (co-ordinated action against domestic abuse) network.
Leadership and skills also came up as vital for organisations to use impact analysis. Without management and board members behind this work it isn’t hard to see how staff can lack the motivation and incentive to use impact analysis to review practice. In the UK Inspiring Impact has also been working on launching a Code of Good Impact Practice to encourage Impact Leadership.
The perceived lack of ‘impact leaders’ and the availability of professionals with the right skills and knowledge were apparent when it came to the discussion of funder involvement in impact analysis. The appropriate involvement of funders and alignment of expectations with investees was identified as a key challenge across all countries.
So, how can we educate and engage with funders and government to ensure this is not a top down conversation?
In Estonia one attendee described the “large dark matter between NGOs and funders” when it comes to communication around impact analysis. At each event attendees coming from both NGO and funder perspectives reiterated this frustration, suggesting there is an imbalance between funder requirements and what is useful and realistic for the organisation in question.
An example of a reporting system that is trying to redress the balance is the new German Social Reporting Standard (SRS). The SRS is a framework for self- assessment for NGOS to report on social impact as well as financial information to funders.The SRS was discussed at a number of the events and met with huge interest especially after hearing that the German Ministry of Social Affairs is now debating whether to adopt this as the primary reporting standard for their work with NGOs.
In Spain at an event organised by the Stone Soup Consulting, Philanthropic Intelligence, Spanish Association of Foundations and SIAA, there was mention of useful practical guidance for funders and social investors. Two examples are The Good Investor and A Practical Guide to Measuring and Managing Impact from the European Venture Philanthropy Association. These guides were outlined as practical resources to guide funders around what is appropriate engagement with investees.
In terms of government involvement, in Bulgaria at an event hosted by the Tulip Foundation a presentation from Nadia Shabani from the Bulgarian Centre for Non-profit Law (BCNL) focused on the need for government to understand the benefits of engaging in impact analysis to help make better social policy decisions. After 15 years working on dynamic legislation, Nadia explained that many social policy models in Bulgaria have been adapted from other countries such as France and the UK. However they often don’t produce the desired outcome as additional effort is need to make the policies work within the Bulgaria context. Impact analysis is something that could enable government and NGOs to identify the barriers to successful implementation of policies, for example in the recent reforms in the child care system.
Similar discussions took place in Ireland and the UK, where local and national government is starting to embed more in depth analysis and evaluation approaches. In the UK the passing of the Social Value Act (2012) requiring ‘public authorities to have regard to economic, social and environmental well-being in connection with public services contracts’ clearly indicates a commitment towards better evidence based policy making. Similarly in Ireland the Irish Government Economic and Evaluation Service (IGEES) has been launched to send economists into government departments to ask key questions around the social impact of their work. It is still an open question as to how much influence these changes will make, but is progress nonetheless.
In Hungary, Estonia and Bulgaria there was also considerable discussion around the involvement of the European Commission, as EU funding contributes to a substantial percentage of funding for NGOs in each country. The challenges identified with this type of funding were that it often leads to over planning of programmes and the use of pre-set indicators which are not useful to the NGOs in their various political, social and economic contexts. In addition there are worries that social impact measurement is a new buzz phrase in Brussels, and that any developments in this area won’t be realistic or appropriate to the needs and organisational capacity on the ground.
Although no one had a quick fix to these challenges there was discussion of how organisations like Ashoka are based in various European countries, but are also able to work in Brussels to influence EU level changes around social enterprises. Many members and attendees suggested that as SIAA develops it might be a very useful vehicle through which analysts and frontline practioners can feedback on the best way forward for impact analysis and measurement issues on a European level. Consultation with analysts and a number of SIAA members is already happening through the GECES Social Impact Measurement Sub-group, to help advise on the European Commission’s Social Business Initiative. Consequently it is not an unrealistic goal for SIAA to play a bigger role in representing analysts at this level.
So, is this real life?
Having covered a number of examples and new developments I would argue yes, this is real life. Although we have a long way to go there are some inspiring case studies emerging across Europe showing that impact analysis can be useful if everyone in an organisation gets behind it. The language emerging is that social impact analysis requires a particular kind of organisational ‘mind-set’ and ethos. Leaders and frontline workers need to be able to engage with analysts and use impact analysis to look at the work of the whole organisation, its strategic objectives and how they fit into the wider system. If done well it can be a real life way for organisations to improve their work in the service of society.
SIAA launched the first Austria Impact Group in November 2012. Three new country Impact Groups are under development in Estonia, Bulgaria, Hungary and Canada with plans for groups in Romania and Portugal in late 2013. Further mapping of the needs in Ireland, Spain and the UK are under way. Get in touch via [email protected] for more information.
This blog was originally written for Alliance Magazine.