How you can improve internal validity in statistics

By Indeed Editorial Team

Published 7 June 2022

The Indeed Editorial Team comprises a diverse and talented team of writers, researchers and subject matter experts equipped with Indeed's data and insights to deliver useful tips to help guide your career journey.

Validity is an important factor when measuring how accurate findings are in research. Internal validity in statistics can help you come to conclusions about the reliability and precision of your data. If you work with statistics or conduct research in your job, learning about this topic can help you with your work. In this article, we explain what internal validity is in statistics, the four different types of validity, what can affect it and how you can improve it.

What is internal validity in statistics?

Internal validity in statistics is the degree to which an investigator or researcher can trust the accuracy of data conclusions, including that it is accurate and free from errors. It is the measure of the reliability and correctness of research. It can show if the appropriate research took place and if there is a real cause-and-effect relationship between the variables involved. If there is validity, then the conclusions and data have a higher chance of being accurate without external factors skewing it.

Validity can increase the confidence you have in trusting your data enough to share the results publicly. It can also help you measure the relationships between variables and identify any cause-and-effect relationships. A lack of validity might mean that variables that were not intentionally included in the research or affected the conclusions made. For there to be internal validity, only the factors involved in the research are the ones that led to the conclusions.

Related: How to become a data analyst

Factors that can affect internal validity

To understand and achieve internal validity in your research, it can be helpful to know the different variables and elements that can affect it. Here are some factors that affect validity:

Changing research methods

Being consistent with the research methods you use is important for achieving validity. Any changes to methods during the research can have an impact. This can include changes to instruments, tools, measures, metrics and other factors. These changes can impact the results and the validity. To increase the accuracy and reliability of your research, it's important to try and keep your methods consistent throughout the process.

Decreased sample size

Participants dropping out of the research study and therefore decreasing the sample size can affect the validity. This can particularly be a challenge for those studies that have multiple sample groups. Different sample populations can change the outcome of the research, which might lead to inaccurate interpretations of correlations. As the sample population you use can affect the internal validation of research, it's important to minimise any changes to it if possible.

Regression to the mean

Regression to the mean happens when outliers cause misinterpretation of data. These outliers might be obvious the first time you measure the data, but the next time they are often closer to the average. This can skew the results. It can also transfer focus, sometimes inaccurately, to sample points that seem closer to the mean. These inaccurate conclusions and interpretations can impact research outcomes and internal validity.

Bias

Bias can happen during research and might take the form of feelings, opinions, or judgements from the research. When a bias is present, it can alter the results of the research and shift them in a certain direction that might not be completely accurate. Bias can make research less sound and also affect internal validity.

Using the wrong research method

If the researcher chooses an inaccurate measurement method for the analysis, it might not lead to the most accurate results. This can be because the method focuses on variables that are not the true causes for the results or because it leads to unreliable data. Using inaccurate research methods can reduce the internal validity of a study.

How you can check the internal validity of research

There are factors you can check to test the internal validity of studies:

  • No confounding variables. An important condition of validity is that there are no extraneous factors or counting variables. If they are present, they can affect the outcomes of your research and skew conclusions.

  • Changes in your variables correlate. If there is a change to the treatment variable (your independent variable) then your response variable (your dependent variable) also changes. When this happens, it can show internal validity.

  • Independent variables precede changes for dependent variables. This means that the variables you treat and alter change before the dependent variables change. This can help you determine a cause-and-effect relationship.

Steps you can take to improve internal validity

There are actions you can take to improve the validity of the research. Here are some examples of what you can do:

Blinding

Using blinding in a study is when participants, and sometimes researchers, do not receive the full and complete information about what is happening. For example, some participants might receive a placebo to avoid biasing their perceptions, behaviours and outcome. The researcher might also be unaware of which participants are receiving a placebo and who are receiving the variable that is being tested.

Randomising sample groups

To avoid systematic biases between groups, you can randomly assign participants to treatment and control groups. This can increase the accuracy of the study as there is less bias over which participants are receiving which variables.

Following study protocols

When the researcher follows specific study protocols and procedures in a study, it can mean that participants go through the same measures. Not following this can increase any effects of biases, for example, conducting the research using different methods for one group compared to another.

Random selection

This is when you choose participants at random or use techniques that ensure they represent the section of the population you want to measure. It can be a helpful way to reduce systematic biases.

The four different types of validity

There are different types of validity, including internal, external, statistically conclusive and construct. Consider these different types:

Internal validity

Internal validity is the extent to which there is a connection and a relationship between the variables involved in the research. It can signify if there is a causal relationship and can increase trust in conclusions that say the factors involved in the research are the reason for affecting the variables. In other words, the data that the researcher measures leads to the results, rather than unknown factors.

External validity

External validity is the extent to which you can apply the findings of the research to people, scenarios and situations outside of the scope of the research. You can generalise or transfer the cause-and-effect to different variables. For example, research done into a group of people who are 50 years old has high external validity when the findings are also relevant to people who are 20 years old.

There are commonalities between internal and external validity. A study with good internal validity can be inappropriate to apply to the real world. Alternatively, you can have a study that happens in the real world, but the results are not trustworthy as you cannot determine which variables had an impact on the outcomes and conclusion.

Read more: Research skills: definition and examples

Statistical conclusion validity

Statistical conclusion validity is the degree to which the conclusions came about because of sound and accurate research and data analysis. It can show if there is a reasonable conclusion about a relationship from our observations. It infers that the data is correct and the appropriate analyses take place.

A lack of conclusive validity can come about because of type I and type II errors. Type I errors happen when the researcher comes to a correlation or a difference when none exists. Type II errors are when the researcher finds no correlation or difference, but one does in fact exist. Statistical conclusion validity involves using the best and most appropriate research methods, tests and measurement procedures.

Related: Analytical skills: definitions and examples

Construct validity

Construct validity is when the evidence supports the hypothesis the research was investigating. It means the conclusions of the data correlate to any concepts or theories the researcher had before starting the investigation. It is important to support the overall validity of the research.

Internal validity example

A researcher wants to discover whether a company that adopts flexible working hours increases job satisfaction for employees. They randomly assign employees to two groups to take part in the experiment. One group (the treatment group) adopts flexible working hours. The other group (the control group) continues with fixed working hours. Both groups fill out a survey at the beginning of the study. The experiment runs for 6 months, at which point participants fill out another survey about their experience. The researcher scores the surveys and concludes that flexible working hours increase employee satisfaction.

But even though the study supports the hypothesis, the researcher cannot remove all other possible extraneous variables that could have affected the treatment group. Examples of extraneous variables can be major life events, career promotions and the strength of the relationship with their manager. The existence of many extraneous variables might mean that the study has a lower internal validity. To increase this, the researcher can remove or control these variables.

Related:

  • How to find critical value in statistics (with definition)


Explore more articles