Impact Reporting - simple, practical and inspiring?

by Jaan Aps, Chairman of Estonian Social Enterprise Network and owner of Stories For Impact

Gathering the resources to build a massive power-plant… for heating a rather small wooden house? That’s how the majority of grass-root level change-makers really feel about setting up a system for evaluating and reporting their impact.  The solution is standardisation combined with one-to-one or group mentoring.

Most social purpose organisations in any country are small or medium sized. Despite looking deceivingly inconsiderable, they are the ones who – as a cluster – have the potential to fulfil the needs of most of the communities and disadvantaged groups. They are the ground for growing large-scale innovative social enterprises. Even those that do not want or succeed to grow tend to remain as indispensable stakeholders in any community-level collective impact initiative.

The following picture outlines:

• some of the problems such organisations have in relation to the three main pillars of the social impact topic (defining, analysing and communicating); and
• the solutions we have worked out and successfully implemented in Estonia.

The standardised format has been tried out by many of the members of the Estonian Social Enterprise Network (as a “soft” membership criterion). Also, some other civil society key stakeholders have opted to use it (e.g. Estonian Association of Large Families).

Compiling the report acts as a catalyst for learning and change for the teams of social purpose organisations. As the photo below demonstrates, we have designed it to be an exciting and challenging process for them!

SIAA’s annual conference “Talking Data: Measurement with a message” will be a great place to learn more about the format and to contribute to its further development.

My vision is to inspire at least some of SIAA’s members and friends to promote a common format that will become the impact accounting disclosure standard for social purpose organisations in their respective countries. Without strong efforts to establish such standard, the impact analysis and communication of (smaller) social-purpose organisations will forever be scattered, unnecessarily confusing and resource-consuming.

One of the main advantages of the format is that it doesn’t dictate how the content is exactly created (as long as the output accords with certain quality standards). I truly hope my workshop at Talking Data, “Simple, efficient and creative solutions for small-scale changemakers”, will spark the discussion – and give participants many valuable hints to be used in their everyday work immediately, even if the larger vision seems too far at the moment.

Below is an example of how the Estonian format looks.

• Why should most of small-scale social purpose organisations have more than one theory of change?
• How can we try out monetization for the organisations with no resources for research?
• In which ways can we support the changemakers so that they really can get the reports completed despite chronic lack of time?

All those questions will be answered at SIAA’s 2014 Annual Conference in November (or earlier, if you write and ask personally).

P.S. Get to know some of the examples of social enterprise reports here and learn about the successful (ongoing) impact journey of Estonian Association of Large Families here.

Jaan Aps runs a social business called Stories For Impact. It provides solutions for identifying, evaluating and communicating societal impact. As an activist, he leads Estonian Social Enterprise Network as a voluntary chairman. Jaan Aps tweets @stories4impact.

This blog was first posted on the Talking Data conference website. Read the original blog here.

Talking Data-to-Data

by Adrian Hornsby, Head of Research at Investing for Good

Everyone is data-greedy in the internet age, but how much of what we consume is “junk data” — bags of factoids, ephemera and particulars that aren’t in themselves false, but that collectively don’t say much more? Data that talks on an individual basis can be great, but often the best conversation one piece of data can have is with another piece of data, and it’s when lots of pieces get involved that higher level meanings start to emerge.

In relation to impact, a lot of attention has been given to how individual organisations collect data and write reports, but perhaps less to how these reports are being read, and — of increasing importance as more and more are generated — how they can be read in conjunction. The image of a grant-officer sitting at year-end-time with a tower of unique impact reports on their desk, and a folder full of unopened spreadsheets in their email, is a problematic one. For one it’s irritating for the grantees who have been required to write and send them, but worse it represents a severe opportunity loss for the sector, as the synthesis of multiple streams of information should be the basis for a larger understanding of what is going on.

In part this is the familiar problem of standardisation (and the associated bête noires of benchmarking, comparability etc.). But given that standardisation remains limited, and that many of the things being measured are just different, there is a more fundamental question as to how non-standard data can be collated, and even aggregated. There is also the more practical one of how this is already happening, and with what results. The use of software is part of the story, and going forward will be instrumental both in recording and assembling large compatible data sets, and in unlocking any correlations embedded within. However the promise of big data and deep learning presents equally an opening for deep biases. As soon as data starts talking to data, it may get the wrong idea about itself.

Adrian Hornsby is Head of Research at Investing for Good and will be presenting a plenary at SIAA’s 2014 Annual Conference, Talking Data: Measurement with a message, on this topic.

This blog was originally published on the Talking Data website and can be viewed here.

SIAA’s July newsletter is here! This month we invite you to participate in the SIAA Challenge 2014, come along to our Talking Data webinar and remind you to get your early bird conference ticket, along with plenty of social impact news and events – take a look.

Interested in receiving a monthly email giving you a round up on all things social impact? Why not sign up for our newsletter.

Impact Measurement – something completely different

Guy Ravid, Senior Social Analyst, Head of Research, Midot, Israel

The reunion of the Monty Python show, celebrated last week, was also screened live here in Tel Aviv at YES Planet. I couldn’t get a ticket to the occasion, so instead I went with my anglophile older boy to a three-day marathon of the groups’ movies at the Tel Aviv Cinematheque. We watched Life of Brian, the Holy Grail and the Meaning of Life, and it was, as I remembered it from my adolescence years, something completely different.

In between one gag and another, my thoughts were drifting to my work. As a Social Analyst, I am simultaneously conducting evaluations for six Non-Profit organizations at any given time, and I was preoccupied with a coming deadline. These NPOs are counting on my reports, since my evaluation of their performance, awarded as Midot’s Seal of Effectiveness, provides them with a prestigious recognition and allows them to join the list of charities approved by Round-Up Israel.

A few days later, everything changed, and my deadline became the least of my worries. Three Israeli teenagers were kidnapped and savagely murdered, a Palestinian teenager was kidnapped and burned alive in a self-proclaimed retaliation by a Jewish settler, and yet another vicious cycle of violence between Israel and the Palestinians erupted. Hundreds of rockets are being launched daily from Gaza onto Israeli cities, and hundreds of missiles and bombs are being dropped over Gaza by the Israeli Air Force. As I write this letter, my family and I are stepping out of the shelter with relief for the third time today.

We live in fear, and our kids are traumatized, but with no comparison to the stress felt by people with disabilities, Israelis and Palestinians alike. Try to imagine a couple who is hearing impaired when the “Red Code” siren is on. Envision a person in a wheelchair who has only 15 seconds to reach to safety in the nearest shelter. Picture a teacher at a special needs school who needs to explain to her students the meaning of the situation and then guide them all safely to a protected area. [Even they have more support and options than the elderly Palestinian lady, her sight almost lost, when informed to leave her house immediately or else she won’t be spared when it is bombed. An alerting siren is not an option in Gaza, and that elderly lady does not enjoy the successful Iron Dome like her Israeli neighbors.]

To support people in need, those whose dependency on others is intensified in times like this, many specialized local charities are shifting their focus to meet urgent needs. Some programs reduce their volume of activity in favor of ad-hoc services, including volunteers providing meals, transportation, and company to the frail and the elderly. Others hire counselors, social workers and therapists to reduce stress and strengthen community resilience. At Aleh Negev, for example, a rehabilitative village for people with severe disabilities, located about 14 miles from Gaza, they are not giving up and continue their routine, allowing residents to enjoy in-house employment opportunities, physical activity, enrichment and leisure, like they always do. However, staff ensures that participants are within 30 seconds from shelter at all times. At the same time, the organization is struggling to retain its staff, hard-working care givers who leave young children at home, with no adult figure to comfort them and keep them safe when the dreadful sound of the siren is heard.

But what does this mean in terms of social impact? Are these organizations still pursuing their goals? How will this affect their results in the long term? One approach suggests that these charities are mobilizing resources and manpower towards pressing missions not included in their plans, sometimes drifting away from their theory of change. Others may say that a high-performing organization is tested in challenging times, and what is more effective than responding to beneficiaries vital needs? Suddenly the term “Impact” gets completely different meaning.

As a recently published report by Médecins Sans Frontières (MSF) teaches us, these dilemmas are not unique to one war zone. The authors of Where is everyone? are acutely aware of the limitations and deficiencies of the international aid response to crises. MSF recognizes that some good work is taking place, and lives are being saved, but much more can, and should be done, to reach those people most in need.

While being occupied with the charities I need to evaluate, my thoughts are also with the Palestinians on the other side of the fence. Through Kiva I lend annually to hard-working Palestinians, mostly women, who struggle to survive and not only make a living, but seek to grow their small businesses in this impossible environment. They too are suffering from a political deadlock, hoping for a better future.

Every day we, at Midot, are trying to find creative ways in which we can help both social investors, and charities, to improve their work and make informed decisions. We urge them to adopt a new thinking, one that relies on evidence-based results. Unlike Cicero, who said “Silent enim leges inter arma“, I believe that our effort should not be limited or diverged even when the drums of war are literally dominating the public sphere.

Being a tireless optimistic, I believe that a worthwhile, encouraging surprise is just around the corner. After all, nobody expects the Spanish Inquisition, do they?

Guy Ravid is Senior Social Analyst and Head of Research at Midot in Israel.

SIAA’s June newsletter is here! This month we invite you to participate in our Talking Data seminar and webinars and remind you to get your early bird conference ticket, along with plenty of social impact news and events. – take a look.

Interested in receiving a monthly email giving you a round up on all things social impact? Why not sign up for our newsletter.

We’ve come a long way

Tris Lumley, Director of Development, NPC

In the summer of 2011, a group of people started planning to work together to help the UK social sector make progress on impact measurement. We were concerned about fragmentation in our approaches, and believed we would achieve more if we coordinated our efforts.

We envisaged a world in which charities and social enterprises would have a clear idea what good impact measurement practice looked like. In this world, organisations would be able to assess themselves against good practice, and identify the tools and resources they needed to make progress. And finally they’d compare their practice with others in their field to learn, and accelerate their progress.

NPC and Substance came together to organise and facilitate an impact ‘summit’ in September 2011, to see if we could establish a collective vision. Out of this summit came a report—thanks to the support of Joe Ludlow at Nesta. It identified five priority areas in which work was needed—on leadership and culture; shared measurement; data, tools and systems; funders, commissioners and investors; and impact measurement support. It also introduced the idea of a cycle of impact practice—how organisations plan, manage, measure, and review their social impact. Out of this report grew the Inspiring Impact programme.

Fast forward nearly three years, and I’m delighted that Inspiring Impact has now launched two important new elements of that original vision.

Measuring Up! is an online tool designed by the Charities Evaluation Services and developed by Substance. It helps organisations to work out how they’re doing against good impact practice. It’s built around the structure of the Code of Good Impact Practice, developed by NCVO through extensive consultation with the sector last year.

The tool is practical, accessible, and makes it easy for organisations to get to grips with the key elements of their impact practice. It’s had rave reviews in testing, and we can’t wait to see how it’s received now it’s available to all—for free.

The Inspiring Impact Hub is an online resource centre for everything you might need to improve your impact practice. If you need guidance on what a theory of change is, you’ll find it there. If you’re looking for software that can help you capture data in your everyday work, you’ll find it in the Hub. And if you’re looking for ways to measure a specific outcome, say well-being, you’ll find it there. Again, it’s all free for everyone to use, and free for tool and resource providers to feature their materials.

The Hub launches with over 200 tools, systems and resources catalogued and accessible to all. But that’s just the start—we want evaluators, tool providers, and others who have developed resources or are aware of other resources to upload them to the Hub. And users can review the tools they’ve used, so everyone gets the benefit of their experience.

Of course there are limits to what online resources on their own can do. We need the concerted efforts of a group of committed partners, and the enthusiasm of others, to embrace them and to help spread the word.

If grant-makers, social investors and commissioners who want social sector organisations to provide good impact data embed these resources in their practice, we’ll build critical mass as people start to explore, use and add to Inspiring Impact. And if those funders also supply their grantees and investees with the financial support needed to build capacity, the dial may really start to shift.

Through the coordination of the Association of Charitable Foundations, a group of funders has already started to get to grips with putting these resources into practice. Inspiring Impact programmes in Scotland (through Evaluation Support Scotland) and Northern Ireland (through Building Change Trust and CENI) are getting the message out far and wide. And NCVO and ACEVO are working with their members to help advance their impact practice.

Join us in building on these foundations—upload tools, share these resources, build them into your work. Together, we can raise the bar on impact practice in the social sector.

This blog was first published on the Inspiring Impact and NPC websites. Click here to view the original post on NPC and here to view on Inspiring Impact.

Tris leads on NPC’s development, including business development, fundraising, partnerships and strategic initiatives. He has helped build NPC’s research, charity analysis framework, measurement team and impact reporting approach, and now focuses on exploring and shaping new initiatives with charities, funders and government. Tris works across the charity sector to advance social impact analysis and leads the Inspiring Impact programme—a partnership across the UK charity sector to help embed useful impact practices in charities and social enterprises. He has led research on charity analysis, campaigning, and within the community sector.

Tris is a trustee of SIAA, a board member of the Charity Commission’s SORP Committee, the ImpACT Coalition, and a trustee of StudentHubs.

Social value matters: The need for a clear language, social data collection and capacity building

by Luis G. Fernandez, Charities Aid Foundation (CAF)

Social impact measurement is important because organisations need to know if they are achieving their mission. More importantly, social impact measurement ‘needs to fly’ because it is the only way to know if organisations are achieving the change that they want to see and the change that makes a difference to their beneficiaries. In fact, social value is ultimately the final goal and the key indicator of success. This is particularly true for organisations working in the philanthropy sector.

Social impact measurement is really far from being complex nowadays. Many tools have been developed and more organisations are ever more familiar with tools that are effective at capturing impact such as social return on investment (SROI). However, why is impact measurement still not motivating many social investors (from high net worth individuals -HNWIs-, philanthropists and corporates)? Ultimately, what needs to happen for impact to be at the core of social investors, philanthropists and donors planning processes?

1. Capacity building

I would say that one of the biggest challenges -if not the most important one- around social impact measurement and indeed social value creation is around capacity building.

Although social impact measurement is not as complex as it used to be in the past, it does require certain skills and capabilities. So, organisations trying to properly address social value creation and impact measurement should really focus in increasing the capacity of their teams, not just by using capacity building exercises but also, by attracting and bringing expertise from other sectors (i.e. academia, overseas development organisations, etc). I keep insisting about this in several global forums: impact is not complex but there is no shortcut. If we want social impact measurement to fly, we need capabilities, skills, social data collection, amongst other important elements that include adhesion to robust principles such as those in SROI.

2. Social data

I couldn’t agree more with a recent blog by Tom Adams from Acumen: “responsible collection of data” or collection of “decision data” is one of the key factors around social impact measurement that really matters. However, although there have been some significant milestones in social data collection, organisations and individuals still privilege financial data collection and data related to performance but not social impact.

If we want social impact measurement to be a constant practice within the philanthropy sector, organisations need to be aware of the importance and benefits of collecting social data related to their projects (e.g. corporate social responsibility –CSR—portfolios, etc.)

3. Performance vs. impact

Many organisations are still articulating their performance in terms of social impact. What they are actually reporting is performance and not social impact. It sounds really basic, but many organisations still talk about ‘how are they doing’ rather than talking about the social impact that ‘they are achieving’. Moreover, this practice is unfortunately embedded in some international standards, indexes, taxonomies available worldwide. Sadly, many organisations use these tools to ‘demonstrate’ their ‘impact’ and/or to benchmark themselves with other peers within their sectors that are also reporting on performance but not social impact.

I think that for impact measurement to fly, organisations need to be able to articulate a general language that practically and efficiently differentiates performance and social impact. More than anything, funders have the greatest responsibility to lead by example in the philanthropic sector.

4. Distractions

Unfortunately, there are many approaches and practices that are not making any favour to social impact measurement: from organisations advising clients using ‘integrated’ approaches to measure ‘impact’ and that are not really helping donors to understand the significance of their investment, to taxonomies that articulate ‘performance’ as ‘social impact’. Having said that, any approach that does not consider principles of economic, environmental and social sustainability is at risk to be considered just performance marketing (and is therefore at risk of being simply marketing and PR strategies).

While these four elements are important other issues such as donors’ level of sophistication, willingness to invest in impact measurement and understanding of the benefits of measurement also play important roles.

Overall, for social impact measurement ‘to fly’, the philanthropy sector needs to lead by example and challenge others to make social value a strategic imperative if we are going to achieve the type of social change we want at scale.

At the Charities Aid Foundation (CAF), when we advise our clients around their donations, we are articulating a clear language that differentiates performance vs. impact. We clearly explain the differences between outcomes and outputs and impacts. We are increasing our capacity to be able to effectively and efficiently use tools such as Theory of Change (ToC) to advise our clients and to clearly identify intermediate and long term outcomes and differentiate them from outputs. We have used the consultancy of the leading UK Think Tank to increase our skills and capabilities to be able to deliver social impact reporting with a robust approach.

We are using social impact measurement to support NGOs and donors in emerging markets to create social change. We are pioneering in some of the different markets in which we operate supporting around 50,000 NGOs worldwide, managing over £2.8bn of assets on behalf of our clients and distributing funds to over 90 countries.

This article was produced for the SROI Network and for the SROI Conference, Social Value Matters, which took place on 12^th and 13^th June 2014.

Luis specialises in economic and social impact analysis across international development; public policies for poverty and gender equality; and, development and humanitarian assistance. He has a strong international development background having previously worked for the UNDP, UNICEF, NGOs and international governments in several countries. Luis expertise also covers monitoring and evaluation, social impact measurement and sustainability reporting. He was awarded with a PhD in Social Policy by the University of Bristol after his research on economic and social impact of structural adjustment in Africa and Latin America. Luis also led the design and implementation of advocacy plans for UNICEF and UNDP in and Africa, East Asia and Middle East.

Introducing Talking Data: Measurement with a message

by Eleanor Radford, SIAA

Last week we announced the theme for our 2014 annual conference and we, at SIAA HQ, are very excited about it. Talking Data: Measurement with a message invites delegates working in the field of social impact measurement and analysis to come together for two days of workshops, hotspots and networking. We will be talking about how we can make our measurements deliver the right message with the most impact.

Building on the success of our 2013 annual conference, Beyond Measurement, it was important to choose a theme that kept the momentum going and addressed some of the unanswered questions. One such question, raised by John Gargani, Gargani + Company, was how do we communicate the impact data we collect? He explains that standards for communicating data are still evolving, which is “driven by a belief that better communication leads to more effective use of impact findings.” It is certainly hard to understate the value of communicating the data we collect well. Whilst we always communicate impact with a purpose, the effectiveness of that communication rests on many other aspects.

When considering these other factors, it is important to think about who we are talking to and how we can influence them. Do we write the same report for external stakeholders and internal decision makers? Of course, what is material will change depending on who the report is targeting. For the SROI Network, materiality is defined as “[determining] what information and evidence must be included in the accounts to give a true and fair picture, such that stakeholders can draw reasonable conclusions about impact.” Yet different reports will influence different readers in different ways. So if we are changing the content of a social impact report for our audience, how do we decide what to include?

Greg Thomson, Charity Intelligence Canada, argued, in a blog for SIAA, the importance of knowing what goes in a report for charities to receive funding. He wrote, “Our report on Social Results Reporting shows that we were only able to find 34% of the information that we would like to see.” He asserts that either they are not reporting externally on the data they collect for internal decision making or they are not collecting the right information. Perhaps, then, it is apt to talk about the extent to which methodology determines what goes in a report and what standards for reporting impact look like.

By bringing together delegates from across the field, we will be able to address questions of materiality, reporting standards and how we ensure that measurements deliver the right message. Talking Data will open up international discussion and inspire our delegates to explore new ideas and hone in on some important issues. Together we can not only get data talking but shouting from the rooftops.

Interested in being involved? Click here to find out about contributing, click here for tickets or contact us for more information.

The Social Impact Analysts Association‘s (SIAA) 2014 Annual Conference, Talking Data: Measurement with a message, will be held in Toronto on November 3rd – 4th in partnership with Social Asset Measurements and Charity Intelligence Canada. Tickets are now on sale!

Where are we heading?

 by Andrew Callaghan, Knightstone Housing

It may sound straight forward but being able to answer the above question isn’t easy! Once you have the answer you can begin to implement an approach to impact measurement.

I have been working in my current role at Knightstone Housing for just over a year and one of my key responsibilities is measuring the social impact of our Individual and Community Empowerment team.

Over the last twelve months I have attended a number of courses on the subject including the School for Social Entrepreneurs (SSE) course on Measuring Social Impact run in association with the new economics foundation (nef), which I would highly recommend.

There are so many resources and approaches to measuring impact, it was a real challenge to find an approach that fitted our requirements. Every organisation and almost all projects are slightly different to each other. My conclusion would be that there is no one approach or tool that you can take off the shelf and run with.

We worked with a consultant to create a report which looked at our current approach to impact measurement and recommendations for how we could embed an approach across our service. This gave a valuable insight from an external perspective. It outlined the lack of a common framework and methodology across the three teams within the service. It also made us re-assess what our long term aims were in order to be clear about what was important for us to measure. Finally, it was a reality check on the time and cost of implementing an approach using external expertise and tools.

Our chosen approach is all about measuring what is important to our stakeholders, spreading the collection of evidence evenly across all members of staff and being realistic about the number of outcome indicators we can collect data on. Our aim is to publish our first Annual Impact Report in May 2015.

Having set out a clear plan has brought about a clarity and energy to drive the project forward. It includes the different monitoring documents, milestones and outcomes we want from the project. Some of the deadlines for each stage are quite tight but the hope is that this will maintain the momentum built up during the project initiation stage.

Finally after a year of research and training seminars, I feel confident about our approach to impact measurement. There are still many hurdles to come along the way but we now know where we are heading, which in turn allows us to measure how close we are to our destination!!

A few of the resources which have been instrumental in creating our approach to impact measurement were:

• Integrated Monitoring written by Sonia Herrero from inProgress
• Inspiring Impact: The Code of Good Impact Practice from Inspiring Impact
• How to Measure and Report Social Impact from the Social Investment Business Group
• Measuring the Social Impact of Community Investment from HACT

Watch this space for updates as Andrew Callaghan’s journey continues.

Andrew Callaghan has been working in the housing sector for over two and a half years.  He is responsible for measuring the social impact of the Individual and Community Empowerment at Knightstone Housing. Interests include the use of Open Data, visualising/mapping data and social impact measurement. Based in Bristol, UK.

SIAA’s March newsletter is here! This month we invite you attend a workshop with SIAA, highlight a report on Social Impact Strategies for Banks and bring you plenty of impact news – take a look.

Interested in receiving a monthly email giving you a round up on all things social impact? Why not sign up for our newsletter.