If there was a type among the 60-strong crowd assembled at the Zollhof Tech Incubator in Nuremberg for the Claude Impact Lab Hackathon, it would be hard to pick it.
Among those that had given up a late July afternoon to see what works were students studying business, technology and engineering, full time-employees taking a break from their day job, and consultants here to find out what’s next.
But one thing that the crowd did share in common was the inkling to solve a challenge in the public interest.
And for those who signed up to the impact track, this could be one of the thorniest in the impact space: how to manage the vast troves of data produced through impact initiatives and turn it into something useful.
After about eight hours, perhaps we got a little closer.
From data to delivery
In the impact sector, impact measurement and management (IMM) is often mentioned but actual understanding can be hard to find. In my own experience, when I was first exposed to the practice, I saw it more of a reporting exercise. But diving in deeper I started to grasp its real value: social enterprises need robust structures to understand and steer their impact, and funders need to properly support them in building those structures.
But IMM remains a major burden for social enterprises, a reality that I’ve encountered in working with social enterprises to build theories of change, reporting structures and measurement tools.
As a newly emerging yet significant field, there is now an opening for practitioners, scholars and support organisations to rethink ways of working and established mantras. In IMM, this often appears in the range of reporting requirements social enterprises have to comply with to satisfy donors and funders. This increases overheads and costs, particularly for cash-strapped organisations.
Additionally, varying definitions mean terms such as outputs and outcomes can be challenging to aggregate and attract funding.
Finally there’s cost. The gold standard is independent evaluation, but there’s simply not enough funding for that to occur at scale. Currently, we mostly rely on independent evaluations the enterprises themselves have done.
Surveying these challenges could be demotivating, but I’m starting to see the outlines of a way through.
Changing tracks
Back at Zollhof, the former customs hall overlooks the spaghetti of rail lines that converge near the centre of Nuremberg. What was a century ago the nexus of measurement and the highways of commerce was once again seeing economic innovation align with precision and value.
But to date AI and impact have followed separate tracks. As global investment in AI has exploded, only a drip feed has been directed toward impact, with less than 1 per cent of global corporate investment in AI directed at products and services built for social impact.
In IMM in particular, a specialised subfield of a sector prone to retreating into its own bubble, I could see this in the questions the participants at the hackathon were asking: “How important is this topic to you?”, “Why does this matter?”, “Why not just use three KPIs across all programmes?”, “Why do the surveys keep changing?”.
These are worthwhile questions, and ones those of us in the impact space could do well to ask ourselves more often, but it also showed what happens when worlds collide and differing perspectives meet. The clarity that comes from going back to basics reveals our own blindspots and sharpens the view on what’s next. It’s exactly these kinds of experiences that we need.
What did we learn, and where next?
At the end of the night, as teams presented their solutions, what stood out to me was the speed at which solutions could be built, prototyped and shared, with working demonstration models built in under a day. We’ve begun a dialogue with one of the teams that tackled the impact challenge and will see how this can be applied to our work.
But these insights from all teams into how rapidly and fundamentally AI can challenge established workflows should go beyond one hackathon. Most importantly, they should flow back to the entrepreneurs, some of whom are experimenting and building their own AI tools to extend their impact.
So here’s three lessons and a question to take forward as we work to integrate AI and IMM for social impact.
- AI can reduce the reporting burden on social enterprises. Every funder has their own requirements and definitions, meaning entrepreneurs spend a large amount of time reporting the same information in different ways. AI could help map indicators across benefactors, pre-fill reports from data enterprises already have, and point us towards a smaller, shared set of indicators. That would free up time entrepreneurs could spend on their actual work
- AI can help in making sense of qualitative inputs. We collect a huge amount of information about what entrepreneurs need to grow, what they’ve achieved and where they’re struggling. AI can help us turn that into a clearer picture of how to support them better. It can also make the contribution of intermediary organisations more transparent, by showing how enterprises actually translated support into their operations.
- AI has its boundaries. AI can surface patterns in what enterprises tell us, but that’s still self-reported data, not evidence of outcomes. It might make gathering outcome data cheaper, for example by analysing beneficiary feedback at scale, but it doesn’t replace rigorous evaluation or human judgment.
A question: what are the limits?
AI is also only as good as the data behind it. In a way, that makes solid IMM foundations even more important: clear definitions, consistent data collection, and fewer but better indicators. And because social enterprises share sensitive information with us, sometimes about vulnerable beneficiaries, we need to be careful about where that data goes and make sure it’s used in their interest.
