What is a biased sample? (Examples and tips to avoid it)

By Indeed Editorial Team

Updated 11 October 2022

Published 6 May 2022

The Indeed Editorial Team comprises a diverse and talented team of writers, researchers and subject matter experts equipped with Indeed's data and insights to deliver useful tips to help guide your career journey.

To conduct research effectively and obtain informative, valuable results requires an appropriate sample. Choosing a sample that reflects the target population, captures the information you require and provides valuable insights is a key priority for researchers. Eliminating bias in the sample is vital in ensuring that results remain accurate and representative. In this article, we look at what a biased sample is, types of samples with bias to consider and steps to avoid bias in future studies.

What is a biased sample?

A biased sample occurs when a study introduces any kind of bias into what would otherwise be a ‘sterile' research environment. There are many different forms of bias, some of which are more subtle than others, that can influence the outcome of studies. A sample with bias is a risk to the validity of any study, which may lead to a need for repetition or results that aren't representative of specific populations and demographics you want to examine.

Related: Research skills: definition and examples

Types of samples with bias

Biased samples aren't always intentional or obvious. It's important to be able to identify and resolve any bias in samples before a study begins to prevent that bias from skewing the results of your study. Some of the most common types of samples with bias you may encounter include:

  • Self-selection: Self-selection occurs when a person directly volunteers or agrees to be a part of a study, which may lead to bias as some demographics could be more willing to volunteer than others.

  • Undercoverage: Undercoverage occurs when a study doesn't include relevant demographics within a study, leading to potential bias by leaving out participants that may be relevant to the results of the study.

  • Non-response: Non-response occurs when a demographic of your sample refuses to take part in a study, leading to bias as your sample may be skewed towards demographics that are available to you.

  • Pre-screening: Pre-screening occurs when researchers interview or screen participants looking to enter into a study, leading to bias by the researcher only selecting participants that fit their goal.

  • Survivorship: Survivorship occurs when researchers select participants they consider a more appropriate sample to support their hypothesis, which may lead to bias as they don't select less attractive participants for the study.

Probability vs. non-probability

Samples with bias typically fall into one of two categories. Both probability and non-probability based bias can have an impact on the validity of your study, but each has a slightly different definition. Here are the key differences between the two:

Bias in probability sampling

Bias in probability sampling typically occurs when everyone in a demographic does not have an equal chance to take part in a study through a selection process. As the study chooses the samples randomly, there's less risk of bias, but this doesn't entirely remove the possibility. Random samples may not accurately reflect the population being studied if the selection frame, i.e. the target list of participants, does not include underrepresented participants.

Bias in non-probability sampling

Bias in non-probability sampling occurs when the selection process that selects the sample in a study isn't random. As researchers select participants in a more direct process or participants agree to take part ahead of time, this can lead to bias. For example, a study at a university in which many of the participants who agree to take part all fall into a specific demographic is non-probability based bias, which can affect the validity of the study.

Related: Research assistant interview questions and example answers

How to avoid sampling bias in studies

Avoiding sample bias in studies is key to achieving reliable and inclusive results from participants. Researchers can take certain steps or invest in certain practices to help reduce the risk of using a biased sample, these include:

1. Define the hypothesis and variables upfront

Identifying the parameters and requirements of a study is a strong starting point in selecting your sample demographic or population. A clear idea of your hypothesis, what you want to test and the process to take for your study can help you define the target audience for your research. Listing the independent and dependent variables you encounter in your study can also support you in picking a representative sample population that covers the full scope of your demographic.

Related: 10 common types of variables in research and statistics

2. Identify the target demographic of the study

Knowing the target demographic for your study is the next step to selecting the right participants. For example, if you plan to create a study examining how many hours of sleep a new parent gets each night, ensure you include parents with a wide range of backgrounds, ages and cultural demographics. It's important that your study accurately represents the type of person you want to learn about.

3. Determine the best way to connect

With a target audience in mind, determining the best way to connect with that audience is the next step. To prevent bias in lesser represented groups, starting with oversampling can be a good way to ensure you aren't sampling based on convenience or survivorship. For example, you could first seek to receive responses from low-income families or single-parent families for your study.

4. Review study questions for bias

Before you begin the study, reviewing or even peer-reviewing the components and questions within your research can help to prevent accidental bias. You can also check study questions throughout the length of the study to ensure the sample has not become biased over time. Check intake questions and forms to ensure they are accessible to your full target demographic.

5. Provide an equal incentive for all taking part

If you plan to carry out a paid study, offering an equal incentive to all taking part can help to reduce bias. For example, in a study about parents, offering free childcare throughout the length of the study can provide an equal incentive for everyone. This kind of incentive can also help reduce bias by providing underrepresented groups with the necessary resources to participate in the study.

6. Ensure an equitable experience for all participants

Providing all participants with the same experience is an effective way to prevent potential bias due to the treatment or behaviour of researchers. Treating all participants the same way and offering them equal opportunity can help to reduce bias. You can evaluate this treatment throughout the study process to ensure there's no bias overtime to any particular part of your demographic.

Examples of samples with bias

Below you can see a few examples of biased samples:

Pre-screening bias example

A researcher is launching a study to look at how office work environments can affect the stress rates of individuals. They send out invites to potential participants and then they meet with each individual to assess their suitability for the necessary time for the study. Only those who have plenty of free time can take part. This bias means that only participants with significant free time, and potentially less stress, are involved in the study.

Undercoverage bias example

A researcher is planning a study that measures how socioeconomics can affect the support available to new parents. They invite local parents to participate in the study via a local online noticeboard and ask volunteers to report on their income for the survey. This survey has a problem with undercoverage bias as they only cover the population from a small area. Parents from a lower socioeconomic household may also be unable to access the internet to take part, leading to even greater undercoverage bias in the sample.

Non-response bias example

A researcher starts a study that examines the effects of stress on students going into their final year of university. They invite students from across the country to participate, and upon volunteering, they ask each student for three hours of their time a week to participate. Very few of the students that applied had responsibilities beyond education. Non-response bias may occur in this sample as students with other responsibilities such as part-time jobs or children may not have the time to take part. The result is a sample with bias that isn't representative of students in all backgrounds and circumstances.

Survivorship bias example

A student researcher is completing a study that looks at students' sleep habits, with a hypothesis that better sleep leads to better grades. They collate all of the volunteer responses to their invites and interview them individually. The researcher then hand-selects students that have better grades and higher levels of sleep. In this example, there's clear survivorship bias, as the researcher selects participants that are more likely to prove their hypothesis correct.

Disclaimer: The model shown is for illustration purposes only, and may require additional formatting to meet accepted standards.

Related:

  • What is non-probability sampling? (Uses, types and benefits)

  • Cross-sectional studies pros and cons (with definition)

  • Cluster vs. stratified sampling: definitions and difference

  • What is a cycle count? (Plus useful methods and examples)

  • What is an affinity bias? (Examples and how to manage it)


Explore more articles