Correlation vs causation in product design and development
By Indeed Editorial Team
Published 26 May 2022
The Indeed Editorial Team comprises a diverse and talented team of writers, researchers and subject matter experts equipped with Indeed's data and insights to deliver useful tips to help guide your career journey.
When you're working in product design and development, correlation and causation may sometimes seem similar. With this in mind, it's worth understanding the difference between these concepts and how to test for causation in your product so that you don't waste resources making unnecessary changes. In addition to product design and development, correlation and causation are also key concepts in fields such as user experience and branding. In this article, we explore the difference between correlation vs causation and different ways to test for causation using hypothesis testing and A/B testing.
Difference between correlation vs causation
When considering correlation vs causation, it's worth defining what these terms mean and why people often confuse them. Correlation and causation refer to two different relationships between factors, events and outcomes. Correlation may be present without causation, so it's crucial to understand how these relationships differ. If two events correlate, there are many possible explanations as to why:
A causes B.
B causes A.
C causes both A and B.
A causes B, but only when D occurs.
There's no causal relationship between A and B.
A causes C, which causes D, which causes B.
Before acting on the relationship between two events, it's necessary to discern whether causation is present and which direction it travels in. Here's a look at the differences between the two:
Correlation describes a relationship between two factors, actions or events. This relationship may be positive, meaning that when one factor increases, the other does as well, or negative, meaning that when one decreases, the other does too. For example, if you're analysing sales figures and comparing them with other data, you might find that when employee bonuses increase, sales revenue also increases. This shows a positive correlation between employee bonuses and sales revenue.
Causation is a relationship between an event and a result where the event causes the result. For causation to exist, there needs to be a clear link between the event and the result. This means that if the event didn't happen, then the result wouldn't have happened either. Using the example above, there is a causal relationship between employee bonuses and sales revenue if you prove that sales revenue only increases when employee bonuses increase.
Correlation doesn't imply causation because correlation only describes a relationship between two factors, while causation explains why the relationship exists. In the above example, it's possible that the correlation between employee bonuses and sales revenue is a coincidence, or there may be other factors that impact both employee bonuses and sales revenue. For example, if you award high employee bonuses around Christmas, the season itself may be the cause of increased revenue instead.
An example of correlation and causation in product design
The example below illustrates how correlation and causation might impact the decisions you make when designing digital products like apps and websites.
Example: You're working on an app for a match-3 style mobile game. New user registrations have stagnated in recent months, so you make some changes to increase the number of new users downloading your app. You reduce the number of adverts that appear during play and design new levels for the app. The month after these changes go live, the number of downloads is up by 20%. This shows a negative correlation between the number of in-game adverts and the number of downloads and a positive correlation between new content and new downloads.
You assume that the changes you made to the game caused the increase in downloads, but there are many other explanations for the correlation you've identified. It's possible that only one of the changes you made had an impact on the download rate. It's also possible that the app's marketing team made changes at the same time, either by increasing advertising spending or generating new content, and that this was primarily responsible for the increase in new downloads.
Why is identifying causation important in product design?
By identifying whether there's a cause-and-effect relationship between two correlating events, you waste less time, money and resources on changes that have no significant impact on the outcome you want to achieve. If you read causation into two correlated variables incorrectly, this is a 'false positive'. False positives are dangerous because they mislead your product design team and result in them:
wasting resources on features or changes that don't matter
focusing on the wrong areas of your product
investing in solutions to problems that don't actually exist
Testing for causation in your product
Knowing how to test for causation in your product lets you conclusively identify whether there's a cause-and-effect relationship present and in which direction it occurs. You can conduct a robust analysis of the correlation between two events to assess any causal relationships in your product analysis. When testing for causation, try to design an experiment that allows you to control other variables and measure the difference in the variables you're testing. The two main types of causation testing are:
Hypothesis testing is a method of testing a hypothesis or an idea to see if it's true. In product design, hypothesis testing allows you to assess whether a change you've made to your product has resulted in the desired effect on users. For example, if you want to increase the number of people who use your app, you might test whether adding a new feature leads to more downloads. To do this, you test both your primary hypothesis (H1) and a null hypothesis (H0), which is the opposite of your primary hypothesis.
Hypothesis testing makes it possible to either prove your hypothesis or disprove your null hypothesis. This type of testing is useful when trying to work out whether a causal relationship between two variables actually exists. As part of your hypothesis testing, you may perform a longitudinal analysis that considers historical changes over time or run a cross-sectional analysis that considers the specific outcomes of a particular factor on different occasions.
A/B testing is a type of hypothesis testing in which a team compares two versions of a product against each other to see which one performs better. A/B testing is commonly used in product design to test how changes to a product impact user behaviour and feedback. For example, you might use A/B testing to compare two versions of a landing page, one with a registration form and one without, to see which landing page results in more new user registrations.
A/B testing is effective when you want to compare two or more different variables, such as product features or marketing strategies. For example, if you want to compare which change to your mobile app increases new user downloads, you might release two iterations of the app, each with one change included. Tracking which version sees the highest increase in downloads helps you identify which change caused the increase. You may then run multiple A/B tests to track the results of different product variations before deciding which changes to implement in the final design.
Tips for identifying correlation and causation in product design
Whether you're designing a product or creating content for a website, try to think carefully about any correlations you identify to prevent yourself from acting upon false assumptions. Until you identify a clear cause-and-effect relationship between two events, assume that a correlation is nothing more than that. Follow the tips below when analysing correlation in your product testing to maximise the usefulness of your product analysis:
Prove a clear connection between variables
Sometimes, even testing for causation doesn't provide irrefutable proof that a connection exists. Aim to look for evidence of a clear and direct connection between two events before taking action in your product design strategy. If you can't prove a clear cause-and-effect relationship, consider undergoing further testing before investing resources in new developments.
Involve the entire team in testing
Successfully testing for causation involves controlling every possible variable so that you can correctly identify exactly which variable triggered a particular effect. Ensure that the entire product team, including developers, designers, marketers and managers, involves itself when testing for causation. This reduces the risk of missing or overlooking key variables during the testing period.
Take advantage of your insights
The insights you gain from identifying causal relationships between two variables help you improve your products in many different ways. You might identify new factors that you hadn't considered before or identify new relationships between different variables that you had previously thought were unrelated. Take advantage of all the insights you gain from causation testing and use them to drive up sales, user engagement, revenue and customer retention.
Explore more articles
- Vesting schedule: types, how it works and how to choose one
- 8 effective sprint planning strategies (with Scrum FAQ)
- What is horizontal integration? (Plus how to implement it)
- A comprehensive guide to pro forma cash flow (with tips)
- How to use HR software (Plus examples of the best software)
- Social media marketing (With benefits, steps and jobs)
- What is CTOR? (Why it matters and how to improve it)
- How to choose the best video editing software for YouTube
- Is sales and revenue the same? A guide to the differences
- What is sales enablement? (With definition and how-to)
- How to recover from a bad day at work (plus remote tips)
- What are shareholders? (Definition, types and FAQs)