Test vs. Control Methodology

Deepanshu Bhalla 6 Comments
Introduction : Test vs. Control Group

In marketing campaigns, marketers send campaign ads to selected customers ("test group") and putting aside the randomly selected some group of customers ("control group") from the campaign. The idea is to determine the effectiveness of the compaign by comparing the sales generated from the "test group" and "control group".


For example, an ecommerce website has segmented customers to send "50% discount on clothing" campaign and they send this campaign to only 50k of them (“test group”), put aside a randomly-selected 5k customers (“control group”) who will not receive it. Once the campaign is over, they will determine the effectiveness of the campaign by comparing the additional revenues generated by the test group with those generated by the control group.

Campaign Performance

The following table provides a comparison between two groups of customers - the "Test" group and the "Control" group. In marketing, the 'Test' group refers to the target audience (users) that receives email or phone call campaigns as part of a new advertising strategy or promotional offer. Whereas, the 'Control' group refers to users who are not part of the campaign or promotional offer.

Variables Test Control
Customer Size 50k 5k
Purchase Rate 15% 10%
Avg Spend Per Customer $150 $125
Total Sales $1,125 $63
Campaign Result : Test vs. Control
Campaign Result : Test vs. Control
Sales Without Campaign

In this scenario, we assume that the company has not sent any campaign or promotional offer to any user.

Variables Without Campaign
Customer Size 55k
Purchase Rate 10%
Avg Spend Per Customer $125
Total Sales $687.5
Test vs. Control Methodology
Test vs. Control Methodology
Sales and Profit from Campaign

The total sales generated as a result of the campaign would be the sum of the sales from both the test and control groups, while the profit generated from the campaign would be calculated by subtracting the sales without the campaign from the total sales.

Total Revenue from Campaign = $1,125 + $63 = $1,187.5
Profit from Campaign = $1,187.5 - $687.5 = $500
Related Posts
Spread the Word!
About Author:
Deepanshu Bhalla

Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 10 years of experience in data science. During his tenure, he worked with global clients in various domains like Banking, Insurance, Private Equity, Telecom and HR.

Post Comment 6 Responses to "Test vs. Control Methodology"
  1. Really useful!! Thank you!

  2. Are their other methods to create the control group where it is not random and the control groups "looks like" the test

    1. you can take out stratified samples for control making sure that it has the same feature distribution as compared to the test group

Next → ← Prev