Lots of charities have access to feedback from their service users, donors or campaigners. Lack of capacity and disconnected data storage can create barriers to analysing it, this means key insights can be missed. Noise Solution knew they had access to “gold dust” hidden within reflection videos shared by their musicians but while the data was rich - existing tools lacked nuance. The problem was clear — could AI help listen to what people are already saying, at scale, and unlock better insight?

“After every session, Noise Solution musicians write a short session report detailing what happened in the session. We wanted to be able to 'see' all of those, all at once, and pull out commonalities. We were interested in what the data was telling us, but the ability to read, hold, and understand it all, all at once, at a macro level, was beyond our ability as humans.”

Start by reviewing the types of feedback your organisation already receives from service users — phone conversations, WhatsApp messages, live chat or transcripts.

Ask yourself:
- What data do we have access to?
- What format is it in?
- Where does it currently live?
- What challenges are we facing when analysing it?
- Do we have the right permissions to process it in this way?

Bring your team together to collectively map what you know, what you don’t know and what you think you know about the challenge. Start a Knowledge Board.

“Your customers tell you what they think about your service every day; in phone records, WhatsApp messages, website chat bots… If you could have a conversation with everyone who uses your services, what might you learn?” Noise Solution

Using tools you are familiar with to run quick experiments so you can see what works well, and where you need additional functionality to analyse your data. Start with the problem you are trying to solve, rather than the tool. Noise Solution started by creating word clouds and doing sentiment analysis to explore themes from the video recordings of young people. The data was rich, but they realised that the process was too time consuming (manually uploading videos). They also observed that the output of this analysis wasn’t in depth enough to be useful to them.

Noise Solution’s hypothesis was that with the support of AI tools, they could unlock insights from natural conversations, turning rich qualitative feedback into measurable data on young people’s wellbeing and personal development. Generative AI analyses transcripts for evidence-based insights into intrinsic motivation and wellbeing, it converts qualitative reflections into numerical scores that can be tracked over time, alongside providing unstructured text insights that can bring reports and funding bids to life.

Ask yourself:

• What is the purpose of your experiment?

• How will you know it is successful?

• What tool will you use?

• Who is responsible for ensuring this is explored responsibly?

Plan your AI experiment

Noise Solution gave young people control and choice about how their data was used. Here are a few examples:

• Clear and simple opt in / out prompts

• Too Long Didn’t Read (TL;DR) video summaries to explain data usage in participant-friendly terms.

• Protect privacy through masking sensitive data

Noise Solution staff checked, reviewed and contextualised the AI outputs by staff. This verification layer builds accountability and prevents AI-generated insights from being misinterpreted or misused. Noise Solution includes a governance stage to review results before sharing or reporting, and is also working on prompt-based bias mitigation strategies to ensure EDI-informed responses.

Looking back, Noise Solution would invest even more in participant co-design from the outset — ensuring that young people fully understand how AI analysis works, and feel confident about how their data is used. They also acknowledge that integrating AI into service delivery requires not just technology, but culture change — supporting staff, funders and stakeholders to understand and trust the insights AI produces.

The work has since led to the creation of a separate spin-out organisation, Transceve, to help other charities benefit from AI engaged data analysis.

“Transceve enhances efficiencies and provides robust impact insights to third sector organisations, using generative AI to analyse conversation data to produce unstructured data insights and discrete measurable variables for trend analyses. By shifting from searching for insights to listening to our data, we transform the feedback process, making it more intuitive, impactful, and live.”