Julie's story

On this page, we’ve pulled together all of the case studies from across the guidance. This will help you see how things can work in practice, as Julie from the Padley Heath Community Centre tries to understand, measure and analyse the wellbeing of people who use the Centre.

Meet Julie from Padley Heath

Julie’s been working at Padley Heath Community Centre for eight years. She’s a great multi-tasker – she’s the centre’s finance manager, office manager, receptionist and occasional support worker. On top of that, she handles the centre’s evaluation activities.

Padley Community Centre

Collecting data at Padley Heath Community Centre

The centre has been running three services for several years:

Adult computer classes

 

A youth club

 

Exercise classes for over-60s

 

Each service has a different funder, and Julie collects data for each one about the service they support, like how many people take part. Traditionally, funders have asked for specific data to do with the reason they chose to support their service, for example for the:

  • adult computer classes the funder wants to know if the classes are improving people’s computer skills
  • the youth club the funder wants to know if the club is reducing antisocial behaviour
  • exercise classes for over-60s the funder wants to know if the classes are improving people’s physical health.
What Julie has noticed

Generally, the data Julie collects shows the activities are doing well. But she has noticed some of the over-60s haven’t really improved their physical health – even though the classes make a big difference to their lives. They tell her all the time that they’re happier, they feel better about themselves and are more connected with their community. But that doesn’t show in her usual evaluations.

Similarly, although she hasn’t measured a drop in antisocial behaviour in her area, she has noticed that young people at the youth club have a greater range of friendships than other people their age. She’s also realised the adult computer classes improve people’s confidence as well as their IT skills.

Julie thinks it would be really worthwhile to find out about all the ‘hidden’ benefits of the activities. Partly to show more of their impact to the funders, but also for the community centre to build on what it does well, and learn what it could improve.

Wellbeing explained

Julie wants to see if the computer classes and the youth club are having the same sort of impact on the participants as the exercise classes are having on the over-60s. If they are, she thinks it will be worth measuring this to show the wider impact of the centre’s services, because it:

  • will help the centre promote the activities
  • will show people the true impact the centre is having on people’s wellbeing
  • might be useful for getting more funding
  • could help the centre improve its services.

So Julie decides to ask some more people at the centre about their experiences.

How to measure it

 Julie knows all three activities at the centre are improving people’s wellbeing in various different ways. She identifies the measures she already uses at the centre (like the number of people who attend or finish the courses) as objective measures. But she knows these other positive changes are to do with how people feel. They’re about subjective wellbeing. She also knows these things are often the first steps to achieving even better overall wellbeing.

Measuring subjective wellbeing

With this in mind, Julie explores subjective wellbeing and how she can measure it for each of the centre’s three projects. From what people have told her, she thinks:


the adult computer classes increase people’s confidence and self-esteem

 


the youth club helps teenagers build relationships and think more positively

 

the exercise classes for over-60s help people increase their social circles and feel more confident in their abilities.

So Julie decides to explore some of the ways she can measure these things – and any other potential differences – and what questions she can ask. 

Your approach

Julie knows from previous projects that it can be hard to get data from people after they have left the project, so she decides that she will implement her survey when new people start and finish activities to try and increase the amount of data she has for analysis.

She also knows that being a young person can be tough, and she doesn’t want to cause any distress to members of the youth club about their social networks. She gets in touch with a local youth organisation to speak to practitioners about how to embed safeguarding in the design of the survey, including guidance on questions and signposting for support if needed.

She decides to create both paper and electronic versions of the survey, so that they can be answered by participants in a way that feels comfortable to them. Both versions will be entered into the electronic version of the survey that only she and one other member of staff can access, and the paper versions disposed of in the confidential waste.

Finally, she creates an information sheet to describe what the questionnaires are, why the data is being collected, how it will be stored and what will happen to the results. Doing this helps her to make sure that participants can give informed consent before they take the survey.

Recommended questions

 Using the ONS questions

Julie looks at the four Office for National Statistics (ONS) questions about wellbeing. Her funders will approve of them because the government uses them. This is especially true for the funder of the exercise class – the local Clinical Commissioning Group – because it’s committed to improving personal wellbeing as part of its strategy.

Julie wants to compare the wellbeing scores of the people using the centre’s services to the average score for people using similar local authority services. She expects the centre’s score to be lower and hopes this will help her make a case for extra funding.

Julie doesn’t want to overload people with too many questions. So she decides to just include the ONS life satisfaction question to get an overall measure of wellbeing. And she’s going to do her survey at the start and end of the centre’s courses of activities to understand how wellbeing changes.

Leaving out the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS)

Julie also considers WEMWBS. She wants to measure the impact of the three services against what matters to people across their whole lives. And as the scale has a psychological focus she decides not to include it.

However, she passes details of the scale to the coordinator of a charity that helps adults with mental health conditions gain confidence through art. She thinks it would be useful for their wellbeing evaluation.

Using social capital questions

Julie decides to use the social capital questions to find out what makes a difference to people’s live across all the centre’s activities. In particular, she wants to know if members of the youth club feel a sense of belonging, and if the over-60s feel supported. 

Write your own questions

Finding out what’s important

Julie considers writing her own questions about the impact of the three services at the centre.

To find out what she should focus her survey questions on, she asks 10 people from each service these questions:

  • Tell me about your life since you joined the group. Has anything changed?
  • How did you feel about yourself and your life before you started coming to the centre?
  • How do you feel now?

Choosing themes for her questions

Lots of people mention feeling more optimistic about the future. For example, people tell her:

  • “Now I can use a computer I feel a lot more positive about my chances of finding work.”
  • “I was really worried about what I was going to do when I’m older, but since I started at the youth club I know I can be successful.”
  • “I was worried about going on holiday with my grandchildren but, now I’m fitter and healthier, I’m looking forward to it.”

So Julie decides to write a question about optimism, to see if she can capture how this has changed for people across the centre’s services.

Developing a pilot question

Julie researches how to design good survey questions. She finds Section 5 of this guide from Leeds University helpful, especially the information on rated responses and question wording. Using that, along with the quotes she collected from people at the centre, Julie writes this pilot question:

How confident do you feel that you’ll be able to do the things you really want to do in the future?

She then adds a scale from 0 to 10 (like the ONS questions) so people can give a rating.

Adjusting the pilot question after feedback

To make sure the question is suitable, Julie asks a few people what they think of it.

She asks if it captures what’s important for their own wellbeing, particularly optimism about their future, and if the wording and layout makes sense to them. People quite like the question, but tell her:

  • “The question has a lot of words.”
  • “The response scale is quite long, and if I respond ‘4’ I wouldn’t know what it meant.”
  • “I am not sure my optimism is just about doing things, it’s also about how I feel about my general life direction.”
  • “What do you mean ‘the future’? Is this next week or in 10 years?”

Julie takes these comments on board and adjusts the question to a statement with different scale:

“I believe I can reach my goals”

Strongly disagree / Disagree / Neither agree nor disagree / Agree / Strongly agree 

Open questions

Julie looks at what open questions to include and decides open questions 1, 2, and 5 (from our ‘Questions you can include’) will be the most useful for her survey. Here’s why:

  • Julie suspects that question 1, alongside her closed questions, will give her more detail on exactly what the centre’s impact is. She also hopes some responses will be good to include in any new bids for funding.
  • Julie thinks question 2 will help her pinpoint what’s having such a good impact, as well as what people don’t like, at the centre. This will show what the centre should do more of and less of to have an even more positive effect.
  • Julie hopes question 5 will confirm the centre is offering support that isn’t available elsewhere. For example, if people in the computer class say they could have gone to a class at their library instead, it may be better for the centre to offer a different service. 

Building a survey

Julie decides she will use these questions for her survey:

  • The ONS life satisfaction question
  • The question she designed on optimism
  • Questions 1,2 and 5 of the open questions

She also makes sure the introductions to her questions and what order they’re in makes things easier for the people taking part.

Testing her survey

Julie decides to test the survey on three people who represent different user groups at the centre. A paper survey isn’t quite right for one of the groups, so she runs an interactive workshop for them instead.

After hearing their feedback, Julie is confident that:

  • she’s not trying to do too much
  • the questions are easy to understand and people will be happy to answer them
  • the questions will help her prove the centre’s impact on people’s wellbeing
  • the workshop approach for one group meets the needs of the people in that group.

So Julie goes ahead with her survey. She tests it on one of the groups first to see how it works in practice and then rolls it out to the other groups.

Analyse and use results

Julie gets the survey results from all her groups and starts to analyse the data.

 

The results from quantitative questions

Julie analyses the results from the question she designed about optimism. The women show more improvement than men when it comes to feeling optimistic. So she realises the centre could do some research about why optimism doesn’t increase as much for men, and find new ways to address that.

The results from qualitative questions

Julie looks at the different responses to the open questions. A lot of the women who report big improvements in optimism are also part of a women’s peer-to-peer support group at the centre.

She decides to ask the men who took part in the survey if they would be happy to discuss the results, and how the centre could help them feel more optimistic. For example, could they start a similar support group for men?


Comparing results for the over-60s

The exercise classes are targeted at a specific age group. So Julie wants to know how people from the classes scored compared to other older people locally and nationally, in relation to the ONS life satisfaction question.

First, she compares the wellbeing scores of other older people living in the local authority area using the benchmark data. The exercise class scores are lower than the average for the local authority area. This is surprising because, from the conversations she had with people in the class, she expected their wellbeing scores to be higher than average.

So Julie then compares the local authority average with the national average, and it’s much higher. The local authority area is comparatively wealthier than the UK overall, so this isn’t so surprising. But Padley Heath isn’t as wealthy as the rest of the local authority area – it’s similar to the UK overall. So Julie decides it’s probably more appropriate to compare her exercise class scores with the national average. And they are higher, so she won’t be able to use this data to help with any funding bids.

To benchmark and report on her evaluation, Julie decides to present her data with both the local authority average and the national average, and explain the differences.                                                                     


Showing the impact of the computer class

As new people join the class, Julie asks them to answer a few quick questions online including demographics, the ONS wellbeing questions and questions on social connection. She makes sure they know how the data will be used. All 50 of the new attendees are happy to fill in the questionnaire.

She compares her results to the benchmark in the UK. She can see from the baseline data that those joining the computer course have lower than average wellbeing and lower than average scores on social connection.

Julie asks people the same questions again after six months into the programme, plus a new open question about what positive changes people have experienced. She reminds the group their responses are private and she won’t use their names. A few of the adults have dropped out of the programme now – but only four, so she’s happy she can use the results.

The results show that there was an increase in life satisfaction and an increase in purpose. She checked using Excel, which shows her this is a significant change.

She wants to be clear why the changes in wellbeing have taken place. This was her thinking behind including the questions and social connection. She wants to understand the intermediate changes that have been taking place in their lives and in turn have led to higher wellbeing.

Julie finds that the adults:

  • who show an increase in wellbeing also show an increase in social connections
  • who started with the lowest scores in social connection show the biggest increase in wellbeing after six months.

From this, Julia realises that:

  • bringing people together through the service could be an important factor in their improved wellbeing
  • she might be able to increase the service’s impact by making sure the centre is targeting people with lower social connection, who may need the service the most.

Julia checks this idea with the answers to her open questions. In their answers, many people – especially people who felt isolated before – mention how positive their new friendships are in their lives, and/or having the space to ‘check in’ with other people. So Julie knows she can start to look at new ways of targeting more isolated people to get them involved in the class.

Improving wellbeing by improving the computer class

Julia also notices an increase in anxiety for the female members of the group. It isn’t a big increase, but she decides to run a short focus group to find out which aspects of the programme could be linked to it.

Some women in the focus group say they worry about keeping up with the pace of the class or finishing the assignment. Julie records this so the centre can make sure women in the next programme can go at their own pace and get enough support.

Other people in the group also mention on their survey that they like the sessions in the room with the window more than the ones in the basement. So Julie notes down that one more thing she can do to improve things is to rethink her booking of the basement room.

What Julie has learnt

Now that Julie has finished her survey she knows how successful the services – especially the computer class – have been when it comes to improving people’s wellbeing. She also knows what changes she can make to try and improve wellbeing even more.

She is going to:

  • look into how the centre can help men feel more optimistic
  • make sure the centre targets more isolated people whose wellbeing would really benefit from the computer classes
  • look at the day-to-day aspects of the services – like what rooms they’re in and the support people get form the centre.