Even if you just want to prove your impact, your survey data can also help you improve what you do to make an even bigger difference to wellbeing. For example, it might show you that you could:

• change your target audience to focus on people who need the most help or who show the greatest improvement from your activities
• change the focus between activities, if your analysis shows some services are more effective than others
• change how you run your services, if your analysis suggests that, for example, certain groups don’t benefit in the same way as others or people prefer certain aspects of a service
• share findings with other organisations in your sector to have a bigger impact together.

What you can improve and how 

Analysing data from different types of question will help you find out what you could change and how:

• Quantitative questions are things like the Office of National Statistics (ONS) questions or the closed questions you designed. They can help you identify areas where you could improve people’s wellbeing. They can also help you see if you’re targeting the people who need your help the      most. For example, if your work focuses on helping people in crisis, you might want to target people who get low scores for domains related to resilience.
• Qualitative questions are the open questions (Section 4). You can use the answers to see what you can do to improve your services and, in turn, people‘s wellbeing.

When you deliver services and activities, your work exists within a wider system. So it’s important to understand what contribution other people or groups make to any changes in wellbeing as well. When you evaluate your activities, ask:

• who and what else is involved in supporting this person or community?
• how can we understand their contribution?

These questions are a great starting point for thinking about attribution – in other words how much credit you can take for changes that a person or community goes through.

Your answers will help you work out:

• if you can make sense of the contribution you make without over-claiming
• if there’s a link between short-term changes and long-term impact, and any existing evidence to justify this link
• where you can draw the line of accountability for a particular project – for example you can help someone prepare for an interview but once they’re in the interview room, you can’t do any more for them and it’s up to them
• how to use short-term information (for example if the number of people taking part in an activity changes over a few months) in longer-term decision making

This is important both for improving your services, as well as proving your impact.

How to identify attribution

Here are some suggestions for how to get an idea of your contribution at the start of your project:

This section is only for people or organisations who:

• are confident and experienced in data analysis
• or have an academic partner who is experienced in more advanced analysis
• or have resources to bring in an expert in analysis.

Ordinary least squares regression

Differences in wellbeing between different groups can sometimes be down to the different characteristics of people within the groups – like age, gender or income. But we often want to test whether wellbeing is different if every other characteristic is equal.

For example, your data may show that people living in areas with lower air pollution have higher wellbeing than people in areas of high air pollution – and it may be a big difference. However, if they live in areas with better air quality, they may also have things like higher income, better facilities and higher-quality jobs. So the difference in wellbeing is showing the combined impact of air quality plus all these other factors.

You can do this with ordinary least squares regression.

What to control for

Wherever possible, try to control for:

• Health and disability (using objective measures where you can)
• Relationship status
• Economic activity or employment
• Income
• Age and age squared
• Gender
• Ethnicity
• Religion
• Highest qualification level
• Dependent children
• Occupational group or status
• Housing tenure
• Interview type
• Region or location

You need to interpret the results of a regression carefully. For example, if you find that ethnicity no longer predicts wellbeing when controlling for income, ethnicity has no direct effect on wellbeing. But ethnicity still affects income, which in turn affects wellbeing.

To find out more about this approach go to:

• Measuring National Well-being – What matters most to Personal Well-being?, Office for National Statistics (2013)
• Measuring National Well-being – What matters most to Personal Well-being? – Regression Results, Office for National Statistics (2013)

(S)WEMWBS: What does a change mean? 

This information is from the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) User Guide Version 2 May 2015:

• Remember you are looking at the total scores across all the questions.
• At a group level, in keeping with other studies, changes of half a standard deviation or more are likely to be important.
• At an individual level a change of three points is regarded as the ‘minimum clinically important difference’.

This has not yet been published for the short version of WEMWBS (SWEMWBS).

Remember you are looking at the total scores across all the questions.

At a group level, in keeping with other studies, changes of half a standard deviation or more are likely to be important. The standard deviation depends on the number of participants so the more participants the smaller the change which can be statistically significant. At an individual level a  change of three points or more in WEMWBS is regarded as the ‘minimumally  important difference’. Below that level improvement is quite likely just to be normal fluctuation. The minimally important difference for SWMEWBS is 2.

Benchmarking is about understanding how your group compares to others in the UK. Here’s how you can compare your scores for the ONS wellbeing questions and WEMWBS.

Benchmarking your results: the ONS wellbeing questions

The ONS wellbeing questions are included on national surveys. So you can compare the wellbeing of people using your services to the average scores for your area or the category of people you’re dealing with.

This can be useful when it comes to:

• understanding the differences in wellbeing in your community, and tracking them over time
• making the case for more funding for your organisation to improve wellbeing levels (if scores are lower than the benchmark scores)
• using it alongside expert judgement for understanding attribution (see ‘How to identify attribution’).

These spreadsheets show the scores for the four ONS wellbeing questions. They have been split to show the differences for protected characteristics including:

• age
• religion
• marital status
• ethnicity
• self-reported health
• economic activity
• reason for economic activity
• reason for part-time work
• local authority and other geographies.

There is a separate Excel spreadsheet for each of the four wellbeing questions so you can compare each one.

There are a few other ways the data has been split to allow for benchmarking:

• ONS4 for over 65s in each Local Authority
• Wellbeing for personal characteristics – including different levels of education
• Most recently, all four wellbeing questions have been split by protected characteristics, including disability and for the first time, sexual identity

These spreadsheets show mean as well as threshold data.

If you would like to research wellbeing from the Annual Population Survey, you can see the End User license version of the dataset at the UK Data Service.

Benchmarking your results: SWEMWBS

The population norms, split by groups, are here.

The SWEMWBS isn’t split up as much as the four ONS questions because the sample size is smaller.

Many organisations measure wellbeing to understand the difference they’re making to wellbeing, and expect their evaluation to show they’re improving wellbeing.

But what if it doesn’t?

Think about why this might be

Try and learn from your data and think about what it tells you. Discuss it with staff and people who use your services. Think about why things haven’t improved, and what this might say about your organisation, your services and the people you support.

• It’s not realistic to expect improvements or high scores for everyone.
• Many people have complex lives and you probably can’t influence the negative impact this can have on wellbeing. But you may be able to help people maintain their wellbeing level, rather than improve it. If people have high anxiety, this could mean you’re reaching the people who need support the most.
• Wellbeing isn’t always stable. Even when someone’s wellbeing is improving overall, there will still be some dips – which may be why a score is lower than you expect.
• Scores can drop for various reasons, unrelated to your intervention. For example, someone could feel more comfortable being honest about how they feel as they develop closer relationships with people.

Qualitative feedback or focus groups can help you understand what’s happening. You may find out that parts of someone’s life are getting better, even though their overall wellbeing scores are lower.

If you’ve made no impact

After carrying out your analysis and delving deeper by using focus groups, you may find you haven’t had an impact on wellbeing. In this case you might need to think about changing your your services to better suit the people you support.