Section 6: Write your own questions

In this section, we give you the pros and cons of using your own wellbeing questions. And a step-by-step process to help you write bespoke questions to add to your survey.

More from Julie

Finding out what’s important

Julie considers writing her own questions about the impact of the three services at the centre.

To find out what she should focus her survey questions on, she asks 10 people from each service these questions:

  • Tell me about your life since you joined the group. Has anything changed?
  • How did you feel about yourself and your life before you started coming to the centre?
  • How do you feel now?

Choosing themes for her questions

Lots of people mention feeling more optimistic about the future. For example, people tell her:

  • “Now I can use a computer I feel a lot more positive about my chances of finding work.”
  • “I was really worried about what I was going to do when I’m older, but since I started at the youth club I know I can be successful.”
  • “I was worried about going on holiday with my grandchildren but, now I’m fitter and healthier, I’m looking forward to it.”

So Julie decides to write a question about optimism, to see if she can capture how this has changed for people across the centre’s services.

Developing a pilot question

Julie researches how to design good survey questions. She finds Section 5 of this guide from Leeds University helpful, especially the information on rated responses and question wording. Using that, along with the quotes she collected from people at the centre, Julie writes this pilot question:

How confident do you feel that you’ll be able to do the things you really want to do in the future?

She then adds a scale from 0 to 10 (like the ONS questions) so people can give a rating.

Adjusting the pilot question after feedback

To make sure the question is suitable, Julie asks a few people what they think of it.

She asks if it captures what’s important for their own wellbeing, particularly optimism about their future, and if the wording and layout makes sense to them. People quite like the question, but tell her:

  • “The question has a lot of words.”
  • “The response scale is quite long, and if I respond ‘4’ I wouldn’t know what it meant.”
  • “I am not sure my optimism is just about doing things, it’s also about how I feel about my general life direction.”
  • “What do you mean ‘the future’? Is this next week or in 10 years?”

Julie takes these comments on board and adjusts the question to a statement with different scale:

“I believe I can reach my goals”

Strongly disagree / Disagree / Neither agree nor disagree / Agree / Strongly agree 

SEE ALL OF JULIE’S STORY ON ONE PAGE

1. Why write your own questions?

Sometimes standard measures can’t fully capture the impact your organisation has on people. That’s because many existing questions:

  • are broad, and don’t pick up changes to specific aspects of wellbeing
  • are not designed for groups who need support answering questions, for example those with dementia or lower levels of literacy
  • use a number scale for the answers, which not everyone is familiar with
  • might not capture aspects specific to your activity or context.

If you can’t collect good, relevant data, you won’t be able to prove or improve the impact you have on the wellbeing of the people you support. So if you design your own questions, you can measure the elements of wellbeing that really matter to them.

2. What you need to know

If you’re going to design your own wellbeing questions, there are a few things to bear in mind:

  • You’ll need to test them on a small group before you include them in your survey to make sure they measure what you want them to measure.
  • They may not be recognised or accepted by funders.
  • People may be reluctant to answer questions they see as very personal.
  • You won’t be able to benchmark your results with other data.

If you’re happy to go ahead, follow our steps to help you.

Step one: find out what’s really changing for people

Your questions need to:

  • measure the elements of wellbeing that are important to people
  • make sense to people.

So you need to listen to people and gather qualitative data about their wellbeing in general, and the impact your organisation has had. This is so you know what to ask about in your evaluation.

Have another look at Section 2 to remind yourself of the different elements of wellbeing and how they fit together.

There are two ways you can get this qualitative data:

Using data you already have 

You may already have case studies from service users, partners, staff and referral organisations. These are valuable because they often describe people’s experiences of your organisation in their own words.

  • Do the case studies explore wellbeing?
  • Do people talk about how they feel about themselves and their lives?
  • Do they talk about whether your organisation has made a difference?

If they do, you can find out what elements of wellbeing to measure, and structure your questions around them.

Collecting new data 

If you don’t have any relevant data already, or you want to make sure your questions help people explain what’s changed in their own words, you can collect new qualitative data.

To do this, you could:

  • do an initial survey with open questions about how people feel about themselves and their lives, and whether your organisation has made a difference
  • run a workshop or do one-on-one interviews – if someone who’s not connected to the organisation can do this, people are more likely to give honest answers.
Step two: identify different elements of wellbeing

Once you have the qualitative data, you can start coding. This is where you look through your notes or transcripts to pick out key themes. Organising your data like this will help you decide what elements of wellbeing are important to people and create your questions.

  1. Make a list of things to look out for before you start coding, for example outcomes from a model of wellbeing. This is called deductive coding. It will help you explain how you decided on themes to explore when you write up your approach. If you notice anything interesting that’s not in the model, you can add it and explain why it’s a new idea. This is called inductive coding.
  2. Look through the data to get a general sense of what people are saying about their wellbeing. Pull out phrases and terms people use to describe their wellbeing.
  3. See if you can group people’s responses into categories.
  4. Note down what’s important to people and how they describe it in their own words.
Step three: develop some pilot questions

You can now use the elements of wellbeing you’ve identified to develop new questions for your survey.

When you design your questions, make sure they:

  • are clear and concise
  • help you capture the level of someone’s wellbeing and how this changes over time – you can do this by asking people to agree or disagree with a statement on a scale rather than just asking for a yes or no answer
  • are not leading questions – so they don’t encourage people to answer a question in a certain way.

Avoiding leading questions

It’s important you don’t put words in people’s mouths or only give them the chance to answer a question in a certain way.

For example, these questions are limiting, and don’t allow people to answer accurately:

  • Have the computer classes made you feel more optimistic about your chances of finding work?
  • How much more optimistic do you feel about your chances of finding work since you started computer classes?

By saying ‘more optimistic’, these questions assume the person is optimistic in the first place, so they can’t say they don’t feel optimistic in their answer.

Step four: test and adjust your questions

You need to make sure the questions you’ve developed reflect what people said about their wellbeing. So go back to people and see what they think of your questions.

Find out if:

  • the questions talk about elements of wellbeing that are important to them
  • the questions are written in a way that makes sense to them
  • they interpret the questions in the same way.

Take what people say on board, and make any necessary tweaks to the wording, the layout and how people can answer the questions.

More from Julie

 Finding out what’s important

Julie considers writing her own questions about the impact of the three services at the centre.

To find out what she should focus her survey questions on, she asks 10 people from each service these questions:

  • Tell me about your life since you joined the group. Has anything changed?
  • How did you feel about yourself and your life before you started coming to the centre?
  • How do you feel now?

Choosing themes for her questions

Lots of people mention feeling more optimistic about the future. For example, people tell her:

  • “Now I can use a computer I feel a lot more positive about my chances of finding work.”
  • “I was really worried about what I was going to do when I’m older, but since I started at the youth club I know I can be successful.”
  • “I was worried about going on holiday with my grandchildren but, now I’m fitter and healthier, I’m looking forward to it.”

So Julie decides to write a question about optimism, to see if she can capture how this has changed for people across the centre’s services.

Developing a pilot question

Julie researches how to design good survey questions. She finds Section 5 of this guide from Leeds University helpful, especially the information on rated responses and question wording. Using that, along with the quotes she collected from people at the centre, Julie writes this pilot question:

How confident do you feel that you’ll be able to do the things you really want to do in the future?

She then adds a scale from 0 to 10 (like the ONS questions) so people can give a rating.

Adjusting the pilot question after feedback

To make sure the question is suitable, Julie asks a few people what they think of it.

She asks if it captures what’s important for their own wellbeing, particularly optimism about their future, and if the wording and layout makes sense to them. People quite like the question, but tell her:

  • “The question has a lot of words.”
  • “The response scale is quite long, and if I respond ‘4’ I wouldn’t know what it meant.”
  • “I am not sure my optimism is just about doing things, it’s also about how I feel about my general life direction.”
  • “What do you mean ‘the future’? Is this next week or in 10 years?”

Julie takes these comments on board and adjusts the question to a statement with different scale:

“I believe I can reach my goals”

Strongly disagree / Disagree / Neither agree nor disagree / Agree / Strongly agree 

Ready to move on?