Section 4: your approach

This section will help you develop your own approach to measuring wellbeing. From essential planning, to the types of questions you can ask for the aspects you want to measure.

More from Julie

Julie knows from previous projects that it can be hard to get data from people after they have left the project, so she decides that she will implement her survey when new people start and finish activities to try and increase the amount of data she has for analysis.

She also knows that being a young person can be tough, and she doesn’t want to cause any distress to members of the youth club about their social networks. She gets in touch with a local youth organisation to speak to practitioners about how to embed safeguarding in the design of the survey, including guidance on questions and signposting for support if needed.

She decides to create both paper and electronic versions of the survey, so that they can be answered by participants in a way that feels comfortable to them. Both versions will be entered into the electronic version of the survey that only she and one other member of staff can access, and the paper versions disposed of in the confidential waste.

Finally, she creates an information sheet to describe what the questionnaires are, why the data is being collected, how it will be stored and what will happen to the results. Doing this helps her to make sure that participants can give informed consent before they take the survey.

 

SEE ALL OF JULIE’S STORY ON ONE PAGE
1. Before you get started

There are a few things you need to do before you start measuring wellbeing:

  • Be clear about what you want to know, how to design your research questions and how to get the answers. Read this NCVO Knowhow Nonprofit guide to help you plan.
  • Familiarise yourself with the different ways to collect data – like surveys, interviews and focus groups. Look at this guide from Research Councils UK for more details.
  • Understand why choosing a representative group of people is important, and how to do it. You also need to know what you can and can’t conclude if you don’t manage to get a good sample of people.
  • Understand why benchmarking, baselining and comparison groups are important. You need to know what the situation was before people took part in your activity, and what would have happened if they didn’t, to see if you made a difference.
  • Make sure you know the difference between outputs and outcomes. Outputs are things like your products and activities, how many people you help and other facts. Outcomes are the effects of your activities – the differences you make. There’s more about this at Knowhow Nonprofit.
  • Involve stakeholders. If you’re measuring wellbeing to prove your impact to stakeholders, they need to be happy with the way you measure it. So get them to sign off your approach before you start.
  • Make sure you can analyse data in Excel using basic formulas. The Office Excel Help pages have articles and how-to guides to help.

Keep it simple

Decide which groups you need to compare and just ask the right questions for them.

2. What to bear in mind when you measure wellbeing

The specifics you want to measure and what to ask

Measuring wellbeing can help you improve what you do. Good data tells you what you’ve achieved and why. So you can find out what to do more or less of to make an even bigger difference in the future.

The questions you ask depend on why you want to measure wellbeing and what you want to measure specifically. For example, you might want to find out:

  • which of your services improves wellbeing the most
  • why certain groups gain more from your programme than others
  • how your services impact on wellbeing now as well as later on in someone’s life.

What you want to know will not only influence your questions but how you collect and analyse the data.

Here are a few prompts about what to measure and why it could be helpful.

What to measure Why measure itWhat it means for your questions and survey design
The wellbeing of people when they join your activity (this is your baseline) It might help you:
• understand if you’re ‘targeting’ in the way you intended
• get a bigger picture of the people who use your service and how the changes your activity is making fit into their lives.
To compare your new starters with others around the UK, you need consistent questions. Like those by the Office of National Statistics (ONS) or the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) (see Section 5).

To understand different aspects of people’s lives, you can ask domain-specific questions, for example about their relationships or work. You can compare the answers with the national results as well (see Section 9).

Using open questions will mean people can answer in their own words.
Whether people’s wellbeing has improved since they started your activity It might help you understand your full impact – how the way people feel or function has changed (‘soft’ outcomes) even if there’s been no ‘hard’ outcome, like getting a job.You’ll need to ask your wellbeing questions before and after your activity, in exactly the same way. This is to make sure you can compare the answers and get reliable results.
How different people improve their wellbeing in different ways. For example, see Sam and Lou’s journeys in section 2)It might help you tailor your services or activities better. For example, you might find the impact you have differs between people
• in different age groups
• from different backgrounds
• with a lower wellbeing to start with.
You’ll need to include questions on demographics (for example, age gender and ethnicity). See Section 8 for some options.
How your impact on wellbeing changes if you run the same activities in different ways or settings.It might help you have a bigger impact on wellbeing. If you find people benefitted more from an activity in a certain location or with specific support, you know what you can do to get better results. You’ll need to use questions about context or be able to split your responses into different groups. For example, by asking things like ‘did you work with a mentor?’ or ‘did you only use the online tools?’ you’ll be able to see which mechanisms are more effective.
Why changes in wellbeing have taken place and / or what changes are related to any improvement in wellbeing (for example better social connection or more confidence)It can help you:
• understand the most important factors for this group
• tailor your activities to improve their wellbeing even more.

You may be able to achieve the same impact in a different way, if you explore these ‘intermediate changes’ – the smaller changes that help people increase their wellbeing even more later on or meet the aims of a specific project.
You’ll need to include questions about the intermediate changes you think are important. For example, changes in social relations or confidence. Have a look at Section 5 and Section 6 for some pointers.

Using open questions can help you understand what’s changed and some of the unexpected reasons why.
The difference between your group and the local or national pictureYou can use this to understand if your services are effective compared to similar organisations in other areas.

If people using your services have lower than average wellbeing, you might be able to use this to make a case for more funding.
You will need to ask consistent questions (like the ONS, WEMWBS or social capital questions in Section 5) so you can compare the answers reliably to national figures.

NCVO Knowhow Nonprofit has all sorts of advice, support and resources to help you plan, do and use your impact evaluation. You’ll also find a range of resources at Inspiring Impact – including datasets, diagnostic tools, and data visualization apps.

3. Some creative approaches to measuring wellbeing

The Arts Observation scale (ArtsObs)

You can use this tool to find out how performing arts activities affect wellbeing in healthcare settings. It’s a way to watch for people’s moods and reactions, and measure the effect the activity has on their wellbeing on a set scale.

The ‘Measuring Wellbeing Creatively’ project

This project is funded through a UCL Grand Challenges: Human Wellbeing Award, and works with people who have communication difficulties because of a brain injury or stroke. It explores how different colours and sensations can help them express feelings and emotions, which may help to measure their wellbeing.

4. When to do your survey

Before and after 

To understand changes in wellbeing, you need to know how people felt before your activity and what would have happened if they didn’t. So ask the same questions before and after the activity so you can compare their answers.

Retrospectively

Sometimes you won’t be able to do a survey at the start of the activity (for example if it’s a long-running activity, and or a casual drop-in). In this case, you can ask people later how they feel things have changed for them during your activity. If you go down this route you may get less accurate or more biased answers than with a before and after survey – and you may not have a robust baseline to compare the answers to. However, this is still a good method if you can’t do before and after surveys.

Follow-up later on

It can be useful to follow up a few months or even a year after a programme to see how long the improvements last.

5. Who asks the questions

If you’re doing a face-to-face survey, it’s important to pick the right person to ask the questions. They need to:

  • be capable and comfortable asking the questions
  • understand why you’re asking the questions and how you will use the answers
  • be able to get reliable responses
  • be able to support and reassure people if the questions are upsetting
  • be able to handle negative responses to potentially difficult questions about emotions.

If you can, ask relevant experts to help – who understand your group and follow your safeguarding procedures.

6. How to choose your questions

Be sensitive to your audience

There may be some wellbeing questions that are not suitable for your audience. For example, for people who have just become involved with your organisation or come to you in a crisis. So use your judgement. You could start with open questions, so they can answer how they like. Or wait until you have a comfortable relationship before you ask anything.

Be realistic 

Don’t include lots of questions if it makes the survey so long that people will give up or you won’t have time to analyse all the answers.

Use the Survey builder in section 8

It will help you create the questions you need for your survey. And NCVO Knowhow Non Profit has lots of extra information if you need more guidance about doing a questionnaire.

Choose carefully: The questions we recommend in Section 5 are based on the latest thinking in wellbeing research and evaluation. But you don’t have to include them all – just what’s useful for your organisation.

Use demographic questions if you want to compare the wellbeing of groups of people with different characteristics. You can identify things like gender, age and ethnicity. In Section 8, we’ve included some from UK government surveys

7. How to ask your questions

Think about the data

Section 5 will help you structure your thinking and decide:

  • what data you need
  • how to collate the data you collect
  • how you can use the data to improve your services and prove your impact.

Then you can start to get creative, and think about the different ways to ask your questions.

Choose your format

Think about your audience and chose the best format for them. For example, you can try:

  • a paper questionnaire
  • an online questionnaire
  • a phone survey
  • more participatory methods

Make your survey clear

Your questions need to be simple and easy to understand for everyone. But some groups may need extra guidance:

  • Think about your audience: once you’ve worked out what questions you want to ask, you could try more creative ways to present them for different groups.
  • Signposting and guidance: think about what you can say or write upfront to make sure people can follow and understand the survey
  • Cultural and language differences: some people may need help with some words. For example, if English isn’t their first language or if some words mean different things for them. For example the term ‘optimistic’ can be interpreted differently across different cultures.Translating questions into other languages could change their meaning, so tread carefully. The ONS is translating their four wellbeing questions into Sylheti and Urdu, and they will be available soon.
  • Using a number scale for answers: some people may be unfamiliar or struggle with using numbers to explain how they feel. You can teach them how, but you need to do that before you ask the questions. And it could be time-consuming and complicated. Is a number scale the best option for your audience?

When to ask wellbeing questions

Some people’s answers can be influenced by questions from earlier in the survey. So try to ask wellbeing questions as early as you can, ideally just after some demographic questions to ease people into it. That way you’re likely to get more accurate answers.

Repeat your survey: remember, to measure the difference in wellbeing between the start and end of your programme, do your survey in exactly the same way before and after – right down to the order of the questions.

 

How to get reliable answers

If you’re after a true picture of how people are doing and what part you played, make sure people can be honest with you.

If people have worked with you before, they may be invested in your work and tell you what they think you want to hear, rather than giving real feedback. You can encourage people to be honest about their experiences if:

  • you ask them questions on their own (rather than in a group setting)
  • the interviewer is impartial and not someone they know already
  • they know their answers will be anonymous.

Avoid bias: just asking questions can influence how people feel and what they do. Some changes in wellbeing may even be down to you asking the questions, rather than your activity.  You can address this, and other types of bias, with larger-scale evaluations. For example, Randomised Control Trials.

8. Data protection and ethics

Get informed consent

This means you need to tell people what you want to use their answers for and get their permission before you start your survey.

Follow the Data Protection Act 

It’s important nobody can identify the people who took part in your survey from your results. So you need to handle everyone’s personal information in line with the Data Protection Act. Find out more at NCVO Knowhow Nonprofit

 

Share what you learn

Sharing your findings can improve the impact of other organisations like yours across the UK, as they learn from your experience. You can send your findings to us at evaluate@whatworkswellbeing.org or connect with other groups on our online forum.

Ready to move on?

  • I understand: