How to Run A/B Split Testing
The first rule of a marketer — «Test».
- Firstly, because it’s easier to sell people what they want.
- Secondly, because it’s only with the help of tests that you can understand whether you made the most of your mailing.
How it happens in theory:
Take a certain percentage of your contacts (you can set it yourself). They are divided into equal parts, and different emails are sent to them. And then look at the result and draw conclusions. I will show you how this is done in practice, but for the time being, please study this important list.
1. What can be tested:
- Subject of email — with or without personalization, whether the question mark at the end of the subject works, the length of the title and whether the call to action works;
- Pre-header (the initial text, which is displayed in some services next to the subject) — whether it works or not.
- Sender — you can test how people react to different addresses and senders. For example, how do your customers open emails from John Smith, Mr. Smith, or John (UniSender)?
- When do your customers read emails? You can easily check the exact time of day when the email was sent.
- How doheadings and subheadings affect performance;;
- Text length — write two different texts about the same offer and compare them to see in which text customers clicked the links more often;
- Design — do you think it’s better to mail to customers in green on a red background? This can also be tested :). But seriously, there is a wide range of things for both experimentation and copywriting.
- Pictures — are they necessary? This can also be checked.
- Whether a unique template helps — if you decide to order a corporate email template, you can test how the indicators have grown.
- What’s better: links or buttons?
- How many links should there be in an email?
- Where should the links be located?
- Whether we should «hide» the buttons or leave them where they are.
- if your web analytics are connected, conversion measurement is the most necessary. We will talk about how to measure the mailing return in the lesson.
And of course, it can be tested in UniSender. Everything except for the reading time of emails and conversion is tested using the Split Test function.
2. How should we test?
Consider an example: Let’s suppose we want to know which button in an email works better, and which one will bring more conversion:
- option – blue;
- option – red.
The button looks like this:
To test the buttons, create a copy of an email:
In the copied email, click the «Change mailing options» tab:
Change the subject of email to «Option 2». We do this for clarity. In the event that you are testing a button, it is not necessary to replace the header. This is so it will not to affect the test result. Click the «Continue» button.
Change the color of the second button in the editor:
The second button is ready. Click the «Save» and «Continue» buttons.
Go to the settings of the «Split test». Click «Conduct A/B test»:
Note: the «Split test» settings can also be accessed from the menu: Campaigns – A/B test reports. On this page you can create a new split test.
The options window opens:
On this page:
1) Specify split-test title.
2) Select contacts list to send a split test:
3) Select options of the email. In UniSender you can test up to 10 options that can be run in a split test.
4) Select the type of split test:
- Split test has not final part.
In this case, the recipients are distributed equally among all the letters participating in the split test. Corresponding campaigns are formed and sent, one for each letter.
- Split test has final part.
Specify the parameter by which we determine the winner:
- With the maximum number of clicks;
- With the maximum number of openings;
- Select the winner manually.
In the example, we are testing the buttons, so we select «With the maximum number of clicks». Which button is clicked more by users.
Other options from the list:
- «With the maximum number of openings» — in the event you are testing the time or subject of the email;
- «Select the winner manually» — if you don’t know what to test. For example, approaches.
Specify the audience.
The point of split testing is to send the best email to a larger audience.
Select a part of the audience:
In our example – we use 20% of the total audience. We send 2 options to this part of the list. 10% will receive the first option and the other 10% the second option.
Contacts are selected randomly. After the split test, look at the mailing results. 80% of the audience will get the best option of the email.
Set the test time. Split tests will be designed for 30 minutes:
It is recommended that you specify at least 3 hours. This is the optimal time for which people open emails. If the audience is inactive or slow (they rarely check their mail), then specify an interval of 1 – 2 days.
5) We can send a letter «Right now» or «Schedule sending» for a different time and date.
6) Set up integration with Google.Analytics/Yandex.Metric, put a tick and turn on the settings:
Ready! The split test is created.
Click the «Launch» button:
Proceed to the split test report from the menu: «Campaigns» — «A/B test reports».
The page displays a list of all split tests. We select the required split test, after which the report and statistics on the letters will be shown:
Congratulations, now you know how to run split testing.